Skip to content

Software Quality and Testing

This course, COMP 10066: Software Quality, provided a comprehensive foundation in the methodologies, tools, and mindsets required to deliver high-quality software. It balanced the theoretical goals of Quality Assurance (QA) with the technical rigor of modern software testing.

Course Overview

The course covered essential concepts from defining quality and understanding the tester's mission, through advanced techniques like Test-Driven Development (TDD) and mutation testing, to practical skills in bug management and UI evaluation. Students learned to think like quality engineers while mastering industry-standard tools and practices.

Major Topics Covered

1. Fundamentals of Quality & Testing

Defining Quality

Understanding how to determine the quality of a product and the relationship between quality and testing. Quality isn't just about the absence of bugs—it's about meeting user needs, performance expectations, and maintainability standards.

The Tester's Goal

Establishing the primary mission of a software tester: to find bugs as early as possible and ensure they are fixed. The earlier a bug is discovered in the software development lifecycle, the cheaper it is to fix. This principle drives all testing strategies.

Quality Assurance (QA)

Implementing systematic monitoring and evaluation to ensure standards are met and processes are improved to prevent bugs. QA focuses on prevention through process improvement, while Quality Control (QC) focuses on detection through testing.

2. Testing Methodologies & Levels

Black-Box vs. Gray-Box Testing

  • Black-Box Testing: Evaluating systems from the outside without knowledge of internal code structure. Testers focus on inputs, outputs, and expected behaviors based on specifications.
  • Gray-Box Testing: Testing with partial visibility into the code, such as inspecting HTML/CSS on a web page. This approach combines the objectivity of black-box testing with the insight of white-box testing.

Unit Testing

Developing automated tests for individual components to verify: - Internal logic correctness - Data coverage (testing various input scenarios) - Code coverage (ensuring all code paths are exercised)

Unit tests form the foundation of a robust testing strategy, catching bugs at the component level before integration.

Website Testing

Addressing the unique challenges of public-facing sites: - Compatibility: Testing across diverse hardware/software configurations - Responsive Design: Ensuring functionality across different screen sizes and devices - Performance: Validating load times, responsiveness, and resource usage - Cross-Browser Testing: Ensuring consistent behavior across Chrome, Firefox, Edge, Safari, and other browsers

3. Bug Management & Life Cycle

Bug Reporting

Mastering the art of writing effective bug descriptions: - Clear Title: Concise summary of the issue - Reproduction Steps: Detailed, step-by-step instructions to reproduce the bug - Expected vs. Actual Behavior: Clear description of what should happen versus what actually happens - Environment Details: Browser, OS, device, and version information - Screenshots/Evidence: Visual proof of the issue - Priority and Severity: Classification to help teams prioritize fixes

The Bug Workflow

Navigating the lifecycle of a bug through various states: - Open: Bug has been reported and needs investigation - In Progress: Developer is actively working on the fix - Resolved: Fix has been implemented - Review: Fix is being verified by QA - Closed: Bug is confirmed fixed and verified - Deferred: Bug is acknowledged but postponed (often due to low priority or resource constraints)

Understanding why some bugs are deferred helps testers prioritize their efforts and communicate effectively with development teams.

Cost of Defects

Analyzing the economic impact of bugs and how the cost to fix them increases significantly as they move through the development lifecycle: - Requirements Phase: Lowest cost to fix (just documentation) - Design Phase: Low cost (design changes) - Development Phase: Moderate cost (code changes) - Testing Phase: Higher cost (code changes + retesting) - Production/Operation: Highest cost (code changes + hotfix deployment + potential business impact)

This concept reinforces the importance of early testing and prevention.

4. Advanced Testing Techniques

Test-Driven Development (TDD)

Adopting a "test-first" model where code is written specifically to satisfy predefined test cases. The TDD cycle: 1. Red: Write a failing test 2. Green: Write minimal code to make the test pass 3. Refactor: Improve code quality while keeping tests green

Benefits include: - Better test coverage - More refactored, maintainable code - Clearer requirements through test specifications - Confidence in code changes

Mutation Testing

Evaluating the quality of a test suite itself by introducing small changes (mutants) to the code to see if the existing tests can detect them. If tests don't catch the mutations, the test suite may have gaps. This technique helps identify weak test cases and improve overall test quality.

Security Testing

Identifying threats to valuable assets, ranking vulnerabilities, and testing for risks such as: - SQL Injection: Testing input fields for database manipulation vulnerabilities - Buffer Overflows: Checking for memory safety issues - Cross-Site Scripting (XSS): Validating input sanitization - Authentication Bypass: Testing access control mechanisms - Sensitive Data Exposure: Ensuring proper encryption and data handling

Security testing ensures applications protect user data and system integrity.

5. The Programmer's Perspective & UI

Readable Code & Standards

Applying industry standards (like Java or C# guidelines) and effective commenting to improve software maintainability. Well-written code is: - Self-documenting through clear naming conventions - Properly commented where logic is complex - Following consistent formatting and style guidelines - Organized with appropriate structure and modularity

User Interface (UI) Testing

Evaluating applications against best practices from major platforms (Apple, Google, Microsoft) and ensuring: - Accessibility: Support for users with disabilities (WCAG compliance) - Usability: Intuitive navigation and clear user flows - Consistency: Uniform design patterns and interactions - Responsiveness: Proper behavior across devices and screen sizes - Visual Design: Adherence to design guidelines and brand standards

Source Control

Utilizing version control systems like Git to: - Manage changes and track revisions - Defend against code loss - Enable collaboration through branching and merging - Maintain project history and rollback capabilities - Support code review processes

Course Learning Outcomes

By the end of the course, students demonstrated the ability to:

  • Refactor programs to follow industry best practices and documentation standards
  • Create software tests, including unit tests and black-box evaluations against written specifications
  • Automate software tests within a development environment to improve efficiency and accuracy
  • Manage the bug lifecycle through professional reporting and resolution tracking

Practical Application: Web Automation Testing with Katalon and Selenium

One of the most impactful practical components of this course involved learning automated web testing using Katalon Recorder and Selenium WebDriver. This hands-on experience demonstrated how theoretical testing concepts translate into real-world automation.

Tool Overview: Katalon Recorder

Katalon Recorder is a browser extension that allows testers to record user interactions with web applications and export them as automated test scripts. It serves as an excellent entry point into test automation, bridging the gap between manual testing and code-based automation.

Key Features: - Record user actions (clicks, form inputs, navigation) - Export to various formats (Selenium WebDriver, C#, Java, Python) - Visual test case management - Easy test case modification and replay

Transitioning to Selenium WebDriver

While Katalon Recorder provides an excellent starting point, transitioning to Selenium WebDriver enables more sophisticated automation capabilities:

  • Cross-browser testing: Run the same tests on Chrome, Firefox, Edge, and other browsers
  • Programmatic control: Dynamic test logic, loops, conditionals, and data-driven testing
  • Integration: Work seamlessly with testing frameworks like MSTest, NUnit, or JUnit
  • Maintainability: Better code organization and reusability

Assignment Example: Multi-Browser Test Framework

In Assignment 5, we built a comprehensive automated testing framework using Selenium WebDriver in C# (.NET 6) to perform cross-browser web application testing. This project demonstrated proficiency in test automation, multi-browser compatibility testing, and dynamic element location strategies.

Multi-Browser Configuration

The framework was designed to run the same tests across different browsers without code duplication:

private static IWebDriver driver;
private const string BROWSER = "CHROME"; // Configurable: CHROME, FIREFOX, EDGE
private const string DRIVER_LOCATION = @"C:\Drivers";

[ClassInitialize]
public static void InitializeClass(TestContext testContext)
{
    if (BROWSER == "FIREFOX")
    {
        FirefoxDriverService service = FirefoxDriverService.CreateDefaultService(DRIVER_LOCATION);
        driver = new FirefoxDriver(service);
    }
    else if (BROWSER == "CHROME")
        driver = new ChromeDriver(DRIVER_LOCATION);
    else if (BROWSER == "EDGE")
        driver = new EdgeDriver(DRIVER_LOCATION);
}

[ClassCleanup]
public static void CleanupClass()
{
    try
    {
        driver.Close();
        driver.Dispose();
    }
    catch (Exception)
    {
        // Ignore errors if unable to close the browser
    }
}

Key Learning: Using Selenium's IWebDriver interface allows the same test code to work with any browser driver implementation, enabling true cross-browser testing with minimal code changes.

Test Lifecycle Management

Proper test structure ensures tests are isolated, repeatable, and don't interfere with each other:

[TestInitialize]
public void InitializeTest()
{
    verificationErrors = new StringBuilder();
}

[TestCleanup]
public void CleanupTest()
{
    Assert.AreEqual("", verificationErrors.ToString());
}

Each test case is self-contained—opening the browser, logging in, performing actions, and logging out—ensuring test independence.

XPath: Mastering Dynamic Element Location

One of the most critical skills learned in this course was using XPath to locate elements in dynamic web pages. This became especially important when working with tables and lists where content changes based on user actions or data.

The Challenge: Dynamic Table Content

When testing web applications, we often encounter dynamic tables where row numbers change based on data. Consider a user management table where new users are added and existing users are deleted. A hardcoded selector like tr[10] would fail when the table structure changes.

Problem Example: 

// This fails if user appears in a different row
Assert.AreEqual("testUser1", 
    driver.FindElement(By.XPath("//div[@id='body']/div/table/tbody/tr[10]/td[3]")).Text);

The tr[10] refers to row 10 of the table. Since the table is dynamic, there's no guarantee that the next time the test runs, the user will end up in row 10 again.

Solution: XPath with Unique Identifiers

Instead of relying on row numbers, we can use XPath to find elements by their unique attributes. When viewing the page source, we discovered that each user row has a unique id attribute on the <td> tag.

HTML Structure: 

<table class="members" width="700">
<tbody>
<tr>
    <th scope="col">ID</th>
    <th scope="col">UserName</th>
    <th scope="col">Password</th>
    <!-- ... more headers ... -->
</tr>
<tr>
    <td id="admin"></td>
    <td>0</td>
    <td>admin</td>
    <td>adminP6ss</td>
    <!-- ... more cells ... -->
</tr>
<tr>
    <td id="testUser1"></td>
    <td>1</td>
    <td>testUser1</td>
    <td>password123</td>
    <!-- ... more cells ... -->
</tr>
</tbody>
</table>

Improved Approach: 

// Uses unique ID attribute - works regardless of row position
Assert.IsNotNull(driver.FindElement(By.XPath("//td[@id='testUser1']")));

Understanding XPath Syntax

XPath (XML Path Language) provides a way to navigate through elements and attributes in an XML/HTML document. There are two main types:

1. Absolute XPath

Starts from the root element and follows the complete path: 

/html/body/div[@id='body']/div/table/tbody/tr[10]/td[3]
- Uses single forward slashes (/) - Must account for every tag in the hierarchy - Very brittle—breaks if any element in the path changes

2. Relative XPath

Starts from anywhere in the document and searches for matching elements: 

//td[@id='testUser1']
- Uses double forward slashes (//) to search anywhere - Can skip intermediate tags - More flexible and maintainable

XPath Strategies for Dynamic Content

Strategy 1: Using Unique Attributes

When elements have unique id or name attributes, use them directly: 

// Find element by unique ID
driver.FindElement(By.XPath("//td[@id='testUser1']"));

// Find element by name attribute
driver.FindElement(By.XPath("//input[@name='username']"));

Strategy 2: Using Text Content

Find elements by their text content: 

// Find link by exact text
driver.FindElement(By.XPath("//a[text()='User Admin']"));

// Find element containing specific text
driver.FindElement(By.XPath("//td[contains(text(), 'admin')]"));

Strategy 3: Iterating Through Collections

For tables with multiple rows, iterate through results dynamically: 

// Get all rows in the table
var cityRows = driver.FindElements(By.XPath("//table[@id='cityTable']/tbody/tr"));

// Verify count
Assert.AreEqual(expectedCount, cityRows.Count);

// Loop through and verify each entry
for (int i = 1; i <= expectedCount; i++)
{
    string cityText = driver.FindElement(
        By.XPath($"//table[@id='cityTable']/tbody/tr[{i}]/td[2]")).Text;
    string provinceText = driver.FindElement(
        By.XPath($"//table[@id='cityTable']/tbody/tr[{i}]/td[3]")).Text;

    Assert.AreEqual(cityName, cityText, $"Row {i} city mismatch");
    Assert.IsNotNull(provinceText, $"Row {i} province is null");
}

Real-World Example: User Creation Test

Here's a complete example from our assignment demonstrating XPath usage in a user creation test:

public void TestCreateUser()
{
    // Navigate to login page
    driver.Navigate().GoToUrl("https://example.com/login.php");

    // Login as admin
    driver.FindElement(By.Id("username")).SendKeys("admin");
    driver.FindElement(By.Name("password")).SendKeys("adminP6ss");
    driver.FindElement(By.Name("Submit")).Click();

    // Navigate to user creation page
    driver.FindElement(By.LinkText("User Admin")).Click();
    driver.FindElement(By.LinkText("Create User")).Click();

    // Create new user
    string newUsername = "testUser1";
    driver.FindElement(By.Id("username")).SendKeys(newUsername);
    driver.FindElement(By.Id("password")).SendKeys("SecurePass123!");
    driver.FindElement(By.Name("Submit")).Click();

    // Verify success message
    try
    {
        Assert.IsTrue(driver.FindElement(By.Id("successMessage"))
            .Text.Contains("User created"));
    }
    catch (Exception e)
    {
        verificationErrors.Append($"Success message verification failed: {e.Message}");
    }

    // Verify user exists in table using XPath with unique ID
    // This is the key XPath example - using relative path with unique identifier
    try
    {
        Assert.IsNotNull(
            driver.FindElement(By.XPath($"//td[@id='{newUsername}']")), 
            $"User {newUsername} not found in table");
    }
    catch (Exception e)
    {
        verificationErrors.Append($"User verification failed: {e.Message}");
    }

    // Logout
    driver.FindElement(By.LinkText("Logout")).Click();
}

Key Learning Points: 1. Relative XPath (//td[@id='testUser1']) is more maintainable than absolute paths 2. Unique identifiers (@id, @name) provide reliable element location 3. Dynamic content requires flexible selectors that don't depend on position 4. Error handling with try-catch blocks ensures tests provide meaningful feedback

XPath Best Practices

  1. Prefer ID and Name attributes over XPath when possible (faster and more reliable)
  2. Use relative XPath (//tag[@attribute='value']) over absolute paths
  3. Leverage unique identifiers for dynamic content
  4. Avoid position-based selectors (tr[10]) unless absolutely necessary
  5. Test XPath expressions in browser developer tools before using in code
  6. Document complex XPath expressions with comments explaining the logic

Error Handling and Verification Patterns

A critical aspect of robust test automation is proper error handling. We implemented an error collection pattern that allows multiple assertions per test while ensuring all failures are reported:

private StringBuilder verificationErrors;

[TestInitialize]
public void InitializeTest()
{
    verificationErrors = new StringBuilder();
}

// In test methods:
try
{
    Assert.AreEqual(expected, actual);
}
catch (Exception e)
{
    verificationErrors.Append(e.Message);
}

[TestCleanup]
public void CleanupTest()
{
    Assert.AreEqual("", verificationErrors.ToString());
}

Benefits: - Collects all verification failures and reports them together - Doesn't stop at the first failure - Provides comprehensive test results - Useful for complex validation scenarios

Reflection: Key Takeaways

This course provided invaluable insights into the world of software quality and testing:

  1. Testing is Prevention, Not Just Detection: The goal isn't just to find bugs, but to prevent them through better processes, early testing, and quality-focused development practices.

  2. Automation Amplifies Impact: Automated tests provide rapid feedback, enable comprehensive regression testing, and free up manual testing resources for exploratory testing and edge cases.

  3. Maintainability Matters: Using relative XPath, unique identifiers, and well-structured test code makes tests resilient to UI changes and reduces maintenance overhead.

  4. Cross-Browser Testing is Essential: Understanding browser differences and testing across multiple browsers ensures consistent user experience across platforms.

  5. Bug Management is a Skill: Effective bug reporting, prioritization, and lifecycle management are crucial for successful collaboration between testing and development teams.

  6. Tools are Means, Not Ends: Whether using Katalon Recorder, Selenium WebDriver, or other tools, understanding the underlying principles of testing is more important than mastering any single tool.

Conclusion

COMP 10066: Software Quality provided a comprehensive foundation in software testing that balances theory with practical application. From understanding the fundamentals of quality assurance to mastering advanced techniques like TDD and mutation testing, the course equipped students with both the mindset and technical skills needed to deliver high-quality software.

The hands-on experience with Katalon Recorder and Selenium WebDriver, particularly the deep dive into XPath for dynamic element location, demonstrated how theoretical concepts translate into real-world automation. These skills are directly applicable to professional software development environments where automated testing is essential for maintaining quality at scale.

The course reinforced that quality is not an afterthought—it's a mindset that should be integrated into every phase of the software development lifecycle. By finding bugs early, writing maintainable tests, and managing the bug lifecycle effectively, we can deliver software that not only works but works well for its users.