Contributing Test Results
Contributing test results is a major part of the work we as a team undertake. Contributing a test result helps assure us that the package, image or hardware we are testing doesn’t have regressions, or bugs.
What types of tests are available to run?
There are 3 basic types of manual testcases you can run and submit results for.
Image testing is intended to test a spun image of ubuntu for functionality via real or emulated hardware. The images that are ultimately produced to install the release of the operating system must pass these tests for release.
Package testing is the manual testing of specific features (test cases) in packages. In addition, these tests can inspect for potential regressions from one release to another.
Hardware Testing is about the manual testing of hardware, using specific images of ubuntu. The goal is to get Ubuntu to work great on as a myriad of hardware by providing results for poorly performing hardware and looking for regressions between images and releases.
When / What do I test?
We normally contribute test results during cadence weeks, special testing events, or as part of testing a daily image.
A cadence week occurs every 2 weeks during the cycle. During this week, milestones containing testcases are posted for testing on the respective trackers. Click here to see the current schedule, and testing needs for each cadence week.
Daily Image testing occurs every day, and contributions are always welcome. Contributing an iso test result is a great way to contribute to ubuntu, and learn about how the qatracker works.
Contributing a test result
STEP 1: Choose your image and testcase
Choose an image of ubuntu you wish to test, and the testcase to go with it. View and follow this guide to help you understand how to download and pick a testcase.
STEP 2: Test the image
You’ve got your testcase and your image downloaded. Now, burn the image to a cd, usb stick, dvd, or even boot into a VM. Once the image loads on the hardware, it’s time to execute the testcase. Follow each of the steps listed, and make sure the expected result happens. If something isn’t right, report it as a bug.
TIP: Zsync saves you from having to download the WHOLE image every time you test. Check it out!
STEP 3: Contribute your result!
Did everything succeed, or did you find a bug? Either way, make sure you fill out the submission form at the bottom of the page. If you have to file a bug, remember to use the ‘ubuntu-bug’ command and the bug instructions link at the top of the page to help.
STEP 4: Now what?
Thanks for contributing! By learning to use the tracker, you can contribute test results for daily images and help squash bugs. But, in addition, you can now participate in the cadence week testing where we look at packages and hardware tests as well! Subscribe to the mailing list to ensure you get notified about the cadence testing events and mark your calendar
I want to help, but I got lost somewhere!
Send an email to the QA Community Coordinator and/or the mailing list, firstname.lastname@example.org. They are happy to help you!
What do I need in order to participate?
How do I run a test?
Tests are run via the various qatracker sites. There is a subdomain for each branch of testing.
Tests can be run by choosing the appropriate site, and then milestone. Each milestone will contain products, and tests intended to be run for each product. You are able to read the test, and after executing, submit the result. The ubuntu QATeam wiki has a series of walkthroughs designed to instruct you on utilizing the qatracker to execute and submit manual testcases.
How do I submit a result?
The qatracker handles the submission of your result. Simply fill out the form below the test result and press submit. A launchpad account is required to submit results.
What happens to my contribution?
If you found a bug, the bug and subsequent resolution for the bug will be tracked inside launchpad. In addition, the bug will be added to the list of previous bugs against the testcase/product in which you found it. This helps to prevent duplicates and to spot regressions. In addition, your result is posted on the results page for the community to see, and ultimately is utilized to confirm the packages and images being tested. For instance, an image will not go to final release without positive results. (or a release note, if the issue encountered cannot be solved).