Archive for November, 2012

Nicholas Skaggs

A glance at Autopilot

So, as has been already mentioned, automated testing is going to come into focus this cycle. To that end, I'd like to talk about some of the tools and methods for automated testing that exist and are being utilized inside ubuntu.

I'm sure everyone has used unity at some point, and you will be happy to know that there is an automated testsuite for unity. Perhaps you've even heard the name autopilot. The unity team has built autopilot as a testing tool for unity. However, autopilot has broader applications beyond unity to help us do automated testing on a grander scale. So, to introduce you to the tool, let's check out a quick demo of autopilot in action shall we? Run the following command to install the packages needed (you'll need quantal or raring in order for this to work):

sudo apt-get install python-autopilot unity-autopilot

Excellent, let's check this out. A word of caution here, running autopilot tests on your default desktop will cause your computer to send mouse and keyboard commands all by itself ;-) So, before we go any further, let's hop over into a 'Guest Session'. You should be able to use the system indicator in the top right to select 'Guest Session'. Once you are there, you'll be in a new desktop session, so head back over to this page. Without further ado, open a terminal and type:

autopilot run unity.tests.test_showdesktop.ShowDesktopTests.test_showdesktop_hides_apps

This is a simple test to check and see if the "Show Desktop" button works. The test will spawn a couple of applications, click the show desktop button and verify clicking on it will hide your applications. It'll clean up after itself as well, so no worries. Neat eh?

You'll notice there's quite a few unity testcases, and you've installed them all on your machine now.

autopilot list unity

As of this writing, I get 461 tests returned. Feel free to try and run them. Pick one from the list and see what happens. For example,

autopilot run unity.tests.test_dash.DashRevealTests.test_alt_f4_close_dash

Just make sure you run them in a guest session -- I don't want anyone's default desktop to get hammered by the tests!

If you are feeling adventurous, you can actually run all the unity testcases like this (this will take a LONG TIME!).

autopilot run unity

As a sidenote, you are likely to find some of the testcases fail on your machine. The testsuite is run constantly by the unity developers, and the live results of commit by commit success or failure is actually available on jenkins. Check it out.

So in closing, this cycle we as a community have some goals surrounding easing the burden for ourselves in testing, freeing our resources and minds towards the deeper and more thorough testing that automation cannot handle. To help encourage this move of our basic testcases towards automation, the next series of blog posts will be a walkthrough on how to write Autopilot testcases. I hope to learn, explore and discover along with all of you. Autopilot tests themselves are written in python, but don't let that scare you off! If you are able to understand how to test, writing a testcase that autopilot can run is simply a matter of learning syntax -- non-programmers are welcome here!
Nicholas Skaggs

A glance at Autopilot

So, as has been already mentioned, automated testing is going to come into focus this cycle. To that end, I'd like to talk about some of the tools and methods for automated testing that exist and are being utilized inside ubuntu.

I'm sure everyone has used unity at some point, and you will be happy to know that there is an automated testsuite for unity. Perhaps you've even heard the name autopilot. The unity team has built autopilot as a testing tool for unity. However, autopilot has broader applications beyond unity to help us do automated testing on a grander scale. So, to introduce you to the tool, let's check out a quick demo of autopilot in action shall we? Run the following command to install the packages needed (you'll need quantal or raring in order for this to work):

sudo apt-get install python-autopilot unity-autopilot

Excellent, let's check this out. A word of caution here, running autopilot tests on your default desktop will cause your computer to send mouse and keyboard commands all by itself ;-) So, before we go any further, let's hop over into a 'Guest Session'. You should be able to use the system indicator in the top right to select 'Guest Session'. Once you are there, you'll be in a new desktop session, so head back over to this page. Without further ado, open a terminal and type:

autopilot run unity.tests.test_showdesktop.ShowDesktopTests.test_showdesktop_hides_apps

This is a simple test to check and see if the "Show Desktop" button works. The test will spawn a couple of applications, click the show desktop button and verify clicking on it will hide your applications. It'll clean up after itself as well, so no worries. Neat eh?

You'll notice there's quite a few unity testcases, and you've installed them all on your machine now.

autopilot list unity

As of this writing, I get 461 tests returned. Feel free to try and run them. Pick one from the list and see what happens. For example,

autopilot run unity.tests.test_dash.DashRevealTests.test_alt_f4_close_dash

Just make sure you run them in a guest session -- I don't want anyone's default desktop to get hammered by the tests!

If you are feeling adventurous, you can actually run all the unity testcases like this (this will take a LONG TIME!).

autopilot run unity

As a sidenote, you are likely to find some of the testcases fail on your machine. The testsuite is run constantly by the unity developers, and the live results of commit by commit success or failure is actually available on jenkins. Check it out.

So in closing, this cycle we as a community have some goals surrounding easing the burden for ourselves in testing, freeing our resources and minds towards the deeper and more thorough testing that automation cannot handle. To help encourage this move of our basic testcases towards automation, the next series of blog posts will be a walkthrough on how to write Autopilot testcases. I hope to learn, explore and discover along with all of you. Autopilot tests themselves are written in python, but don't let that scare you off! If you are able to understand how to test, writing a testcase that autopilot can run is simply a matter of learning syntax -- non-programmers are welcome here!
Ara

Ubuntu Certification 101 – BIOS

Last week, at UDS, I was asked several questions about how Ubuntu Certification worked. I decided I would start a set of blog posts about several aspects of the certification programme, for people willing to know a bit more about how we test and what.

One of the most confusing bits about Ubuntu Certification are BIOS options. In several ocassions I was asked the questions whether UEFI (with and without Secure Boot) was going to be added as part of Ubuntu Certification or not.

The answer is both yes and no. I will explain it a bit more in detail.

In Ubuntu Certification we don’t test BIOS options, nor we change them. We try to install Ubuntu with what it comes with the system from factory, and it needs to work up to a certification level using those options. That means that if it is using Legacy BIOS, we will be testing Legacy BIOS, if it comes with UEFI without Secure Boot, that is what we will be testing. If it comes with Secure Boot enabled, Ubuntu will need to install out of the box with the keys already in that system.

From 12.10 we have started to parse the BIOS information as part of the testing process:

We will be updating the public certification site to show that information there as well.

Same thing for BIOS options. We test with factory default options, and only those. The goal of doing this is that Ubuntu Certified systems should provide and out-of-the-box Ubuntu experience. In some cases, tweaking BIOS options can improve the situation for non working systems, but that is something we don’t want to acept for Ubuntu Certification.