Testing versus Checking

I stumble upon this blog post Testing vs. Checking by Michael Bolton which I found very interesting and a fundamental for software testers.  I tried to summarize it and place it in a table to see the distinction clearly.

Checking	Testing
confirming existing beliefs	finding new information
a process of confirmation, verification, and validation	a process of exploration, discovery, investigation, and learning
focused on making sure that the program doesn’t fail	focused on “learning sufficiently everything that matters about how the program works and about how it might not work.” – James Bach
We are checking when: * we’ve made a change to the code and we want to make sure that everything that worked before still works. When we have an assumption that’s important, we check to make sure the assumption holds. * developers write and modify their code, creating automated routines that they run frequently to check to make sure that the code hasn’t broken	We are testing when: * we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated * when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before
A check provides a binary result—true or false, yes or no	A test has an open-ended result
is all about asking and answering the question, “Does this assertion pass or fail?”	is about asking and answering the question “Is there a problem here?”
checking can prove the presence of bugs, but not their absence	the process of finding out whether our checks have been good enough
machine-decidable and are, in and of themselves, value-neutral	requires human, which makes it a sapient process
when done by a programmer, is mostly a quality assurance practice. The quality assurance angle: a programmer helps to assure the quality of his work by checking it.	is not quality assurance, but acts in service to it; we supply information to programmers and managers who have the authority to make decisions about the project.
	“questioning a product in order to evaluate it.” –James Bach. Evaluation of a product doesn’t assure its quality, but it can inform decisions that will have an impact on quality.

 

Checkers	Testers
A person who needs a clear, complete, up-to-date, unambiguous specification to proceed	Does not require the certainty of a perfect specification to make useful observations and inferences about the product
A person who needs a test script to proceed	His job is to discover information; often that information is in terms of inconsistencies between what people think and what’s true in reality
A person who does nothing but to compare a program against some reference	A tester’s task might be to reveal problems that occur when our excellent code calls buggy code in someone else’s library, for which we don’t have a specification

Though people may or may not agree with this, it made me understand where to draw the line. =) Thanks Michael!

Helpful Unix Commands

Below are the helpful Unix commands I learned when we handled SHINE project, where part of our QA task is to checkout/update the latest SVN codes, run maven and jetty in order for us to setup a local build and run our Selenium scripts.

1. ps aux – ps short for “process status” displays all the currently-running processes in the terminal

aux is an option where:
“a” lists all processes on a terminal, including those of other users,
“u” adds a column for the controlling user for each process, and
“x” lists all processes without controlling terminals

2. grep – searches files or string that matches the given expression

e.g.
grep “id=inputaddress” *.pl – searches all files with “.pl” extension and returns files with string “id=inputaddress”

3. pipe “|” – allows you to execute number of commands in sequence

e.g.
ps aux | grep jetty – displays all the currently-running processes in the terminal and filters the jetty process

output:
exist     7712  0.0  0.0   8952   876 pts/0    S+   18:21   0:00 grep –color=auto jetty

4. kill  – sends termination signal, which requests that the process exit.

In our project we frequently used the command “kill -9″ everytime our scripts hang and Google taught me that there are several ways to kill a task. Based on our output in item #3 let’s try to kill the jetty process with process ID 7712:

e.g.
-9 and -KILL are options to send SIGKILL, a signal sent to a process to cause it to terminate immediately
kill -9 7712
kill -KILL 7712

-15 and -TERM are options for SIGTERM, a signal sent to a process to request its termination. This signal can do useful cleanup operations (e.g. saving configuration information to a file) before quitting
kill -15 7712
kill -TERM 7712

5. dpkg -L {package} – List where files/package are installed

e.g.
dpkg -L maven2 – list where Maven is installed

Perl-Selenium Helpful String Manipulations

Below are the string functions I found helpful during scripting in my current project using Selenium in Perl language:

Split(/PATTERN/,EXPR)
- Splits the string EXPR into a list of strings based on given PATTERN and returns that list.

# Scenario: Clicking Email link opens new window. Verify new location/URL displayed is correct
# Email element: <a onclick=”window.open(‘http:test.url’,'EMAIL’,'width=730,height=450,status=yes,toolbar=no,menubar=no,location=no’);” style=”color: rgb(38, 6, 2); font-size: 11px; text-decoration: underline;” href=”javascript: void(0);”>Email</a>

my $href = $sel->get_attribute("xpath=//html/body/div[2]/div[2]/div[2]/table/tbody/tr/td[2]/div[3]/a/\@onclick");
# $href = "window.open('http:test.url','EMAIL','width=730,height=450,status=yes,toolbar=no,menubar=no,location=no');"
my @url = split(/'+/, $href);
# @url = ['window.open(' , 'http:test.url' , ',' , 'EMAIL' , ',' , 'width=730,height=450,status=yes,toolbar=no,menubar=no,location=no' , ');' ]

Substr(EXPR,OFFSET,LENGTH)
- Extracts a substring out of EXPR and returns it based on defined OFFSET and LENGTH

# Scenario: Get the numeric value from the Total distance string element

my $str = $sel->get_text("css=div.olPopupContent>div.mainbubblecontent>div.mainbubbletabcontent>div.activetabcontent>div:nth-child(5)");
# $str = "Total distance: 100.25 miles";
my $distance = substr($str,16,5);
# $distance = '100.25';

Match (m/PATTERN/)
- match a string with a regular expression pattern

my $str = "Total distance: 100.25 miles";
 if ($str =~  m/ \d*.\d* miles/) {
   print "Pass";}
 else{
   print "Fail";}

Cmp_ok( $got, $op, $expected, $test_name )
– Test::More function that allows you to compare two arguments using any binary perl operator.

my $str = $sel->get_text("css=#panel > #collection_maneuvers > thead > tr > th");
# $str = "Total distance\: 100.25 miles"
cmp_ok($str, "=~", m/Total distance\: \d*.\d* miles/, "Verify estimated mileage is displayed");

Like( $got, qr/expected/, $test_name )
– Another Test::More function that evaluates any expression against a regular expression

my $str = $sel->get_text("css=#panel > #collection_poi > thead > tr.poi > th");
# $str = "5 locations found in your area"
like($str, qr/\d* locations found in your area/, "Verify number of locations is displayed");

Also came across this Perl documentation link (http://perldoc.perl.org) which is direct and comprehensive.

Agile Testing

I had the opportunity to attend one of the trainings on Agile Testing last August, 2011 conducted by CHEQ Systems. Inspired by this training I’ve compiled some of the basics of Agile Testing that I’ve learned:

Definition

Agile Testing is an iterative and incremental(evolutionary) approach to software development performed in a highly collaborative manner by self-organizing teams within an effective governance framework with “just enough” ceremony that produces high quality software in a cost effective and timely manner which meets the changing needs of the stakeholders.

Agile Values

1. Individuals and interactions over processes and tools – emphasis on people and group interactions.
2. Working software over comprehensive documentation – documentation is not a major goal but a supporting medium for the actual product – the software.
3. Customer collaboration over contract negotiation – involving the customers in the software development rather than on detailed contract negotiation.
4. Responding to change over following a plan – development progresses in response to user feedback, rather than as reaction to a fixed plan, but it does not mean that there is no plan.

Criteria in determining an Agile team

1. Highest priority is to satisfy the customer
2. Welcomes change
3. Deliver working software frequently
4. Business people and developers work closely together daily
5. Build projects around motivated individuals
6. Face to face communication is the best
7. Working software is the primary measure of progress
8. Promote sustainable development
9. Continuous attention to technical excellence and good design enhances agility
10. Simplicity
11. The best architectures, requirements and design emerge from self-organizing teams
12. Introspective – teams should regularly review itself and its processes for improvement

Criteria to determine an Agile Team

1. There is an active participation of stakeholders/customers
2. Developers do Test-Driven Development approach and do regression testing
3. Team is producing high quality working software on a regular basis
4. Team is highly collaborative and self-organizing
5. Team is improving

That’s all for now, I’ll try to create a post for Agile Testing Strategies that the training highlighted.

2010 in Review

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Wow.

Crunchy numbers

About 3 million people visit the Taj Mahal every year. This blog was viewed about 36,000 times in 2010. If it were the Taj Mahal, it would take about 4 days for that many people to see it.

In 2010, there were 4 new posts, growing the total archive of this blog to 43 posts. There were 3 pictures uploaded, taking up a total of 530kb.

The busiest day of the year was March 18th with 201 views. The most popular post that day was Creating Firefox profile for your Selenium RC tests.

Where did they come from?

The top referring sites in 2010 were luhman.org, google.co.in, codebetter.com, google.com, and passionatetester.com.

Some visitors came searching, mostly for selenium firefox profile, selenium rc firefox profile, firefox profile selenium, selenium profile, and jmeter proxy.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

1

Creating Firefox profile for your Selenium RC tests February 2009
49 comments

2

JMeter 101: Using HTTP Proxy to Record your Test March 2009
19 comments

3

Jmeter: Add “Cookie Manager” June 2009
8 comments

4

JMeter: Run scripts from the console October 2009

5

Jmeter: Statistics Aggregate Report January 2010
4 comments

Selenium RC to test unsecured connection HTTPS

It’s inevitable for software testers to run test in an environment with self-signed SSL certificates. This became one of my dilemma when trying to run my Selenium scripts in an HTTPS environment and was always prompted with “This Connection is Untrusted” error.

I have an existing Firefox Profile solely for Selenium, if you don’t have one, you can check this post.  Was able to resolve this issue by doing the following:

1. Launch Profile Manager by typing “firefox -ProfileManager -no-remote” in your terminal (Linux user)
2. Select Selenium profile then Start Firefox
3. Access your web application URL in HTTPS
4. Accept the SSL Certification:
• Click “I Understand the Risks”
• Click “Add Exception”
• Click “Get Certificate”
• Make sure “Permanently store this exception” tickbox is checked
• Click “Confirm Security Exception”
5. After successfully directed to the web application page, close Firefox
6. Go to Selenium Profile folder ( in my case /home/girlie/.mozilla/firefox/selenium )
7. Delete all files except for cert_override.txt and cert8.db files.

From here on, I rerun my Selenium scripts and didn’t encountered the “This Connection is Untrusted” error anymore

How to live before you die –Steve Jobs

Most of us have benefited from the brilliance and countless innovations made by the Apple man itself Steve Jobs.

Jobs, former chief executive officer and co-founder of Apple died on October 5, Wednesday (October 6 in the Philippines ) at the age of 56. He was first diagnosed with an unusual form of pancreatic cancer in 2004 and needed to undergo liver transplant in 2009.

Today as we mourn on Jobs’ death,  I was reminded by his memorable 15-minute commencement address delivered on June 12, 2005, at the Stanford University in Palo Alto, California.

Thank you Steve Jobs for making our world a better one.

Here is the full transcript of Jobs’ speech:

I am honored to be with you today at your commencement from one of the finest universities in the world. I never graduated from college. Truth be told, this is the closest I’ve ever gotten to a college graduation. Today I want to tell you three stories from my life. That’s it. No big deal. Just three stories.

The first story is about connecting the dots.

I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why did I drop out?

It started before I was born. My biological mother was a young, unwed college graduate student, and she decided to put me up for adoption. She felt very strongly that I should be adopted by college graduates, so everything was all set for me to be adopted at birth by a lawyer and his wife. Except that when I popped out they decided at the last minute that they really wanted a girl. So my parents, who were on a waiting list, got a call in the middle of the night asking: “We have an unexpected baby boy; do you want him?” They said: “Of course.” My biological mother later found out that my mother had never graduated from college and that my father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents promised that I would someday go to college.

And 17 years later I did go to college. But I naively chose a college that was almost as expensive as Stanford, and all of my working-class parents’ savings were being spent on my college tuition. After six months, I couldn’t see the value in it. I had no idea what I wanted to do with my life and no idea how college was going to help me figure it out. And here I was spending all of the money my parents had saved their entire life. So I decided to drop out and trust that it would all work out OK. It was pretty scary at the time, but looking back it was one of the best decisions I ever made. The minute I dropped out I could stop taking the required classes that didn’t interest me, and begin dropping in on the ones that looked interesting.

It wasn’t all romantic. I didn’t have a dorm room, so I slept on the floor in friends’ rooms, I returned coke bottles for the 5¢ deposits to buy food with, and I would walk the 7 miles across town every Sunday night to get one good meal a week at the Hare Krishna temple. I loved it. And much of what I stumbled into by following my curiosity and intuition turned out to be priceless later on. Let me give you one example:

Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every drawer, was beautifully hand calligraphed. Because I had dropped out and didn’t have to take the normal classes, I decided to take a calligraphy class to learn how to do this. I learned about serif and san serif typefaces, about varying the amount of space between different letter combinations, about what makes great typography great. It was beautiful, historical, artistically subtle in a way that science can’t capture, and I found it fascinating.

None of this had even a hope of any practical application in my life. But ten years later, when we were designing the first Macintosh computer, it all came back to me. And we designed it all into the Mac. It was the first computer with beautiful typography. If I had never dropped in on that single course in college, the Mac would have never had multiple typefaces or proportionally spaced fonts. And since Windows just copied the Mac, it’s likely that no personal computer would have them. If I had never dropped out, I would have never dropped in on this calligraphy class, and personal computers might not have the wonderful typography that they do. Of course it was impossible to connect the dots looking forward when I was in college. But it was very, very clear looking backwards ten years later.

Again, you can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.

My second story is about love and loss.

I was lucky — I found what I loved to do early in life. Woz and I started Apple in my parents garage when I was 20. We worked hard, and in 10 years Apple had grown from just the two of us in a garage into a $2 billion company with over 4000 employees. We had just released our finest creation — the Macintosh — a year earlier, and I had just turned 30. And then I got fired. How can you get fired from a company you started? Well, as Apple grew we hired someone who I thought was very talented to run the company with me, and for the first year or so things went well. But then our visions of the future began to diverge and eventually we had a falling out. When we did, our Board of Directors sided with him. So at 30 I was out. And very publicly out. What had been the focus of my entire adult life was gone, and it was devastating.

I really didn’t know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down – that I had dropped the baton as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and I even thought about running away from the valley. But something slowly began to dawn on me — I still loved what I did. The turn of events at Apple had not changed that one bit. I had been rejected, but I was still in love. And so I decided to start over.

I didn’t see it then, but it turned out that getting fired from Apple was the best thing that could have ever happened to me. The heaviness of being successful was replaced by the lightness of being a beginner again, less sure about everything. It freed me to enter one of the most creative periods of my life.

During the next five years, I started a company named NeXT, another company named Pixar, and fell in love with an amazing woman who would become my wife. Pixar went on to create the worlds first computer animated feature film, Toy Story, and is now the most successful animation studio in the world. In a remarkable turn of events, Apple bought NeXT, I returned to Apple, and the technology we developed at NeXT is at the heart of Apple’s current renaissance. And Laurene and I have a wonderful family together.

I’m pretty sure none of this would have happened if I hadn’t been fired from Apple. It was awful tasting medicine, but I guess the patient needed it. Sometimes life hits you in the head with a brick. Don’t lose faith. I’m convinced that the only thing that kept me going was that I loved what I did. You’ve got to find what you love. And that is as true for your work as it is for your lovers. Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle. As with all matters of the heart, you’ll know when you find it. And, like any great relationship, it just gets better and better as the years roll on. So keep looking until you find it. Don’t settle.

My third story is about death.

When I was 17, I read a quote that went something like: “If you live each day as if it was your last, someday you’ll most certainly be right.” It made an impression on me, and since then, for the past 33 years, I have looked in the mirror every morning and asked myself: “If today were the last day of my life, would I want to do what I am about to do today?” And whenever the answer has been “No” for too many days in a row, I know I need to change something.

Remembering that I’ll be dead soon is the most important tool I’ve ever encountered to help me make the big choices in life. Because almost everything — all external expectations, all pride, all fear of embarrassment or failure – these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.

About a year ago I was diagnosed with cancer. I had a scan at 7:30 in the morning, and it clearly showed a tumor on my pancreas. I didn’t even know what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than three to six months. My doctor advised me to go home and get my affairs in order, which is doctor’s code for prepare to die. It means to try to tell your kids everything you thought you’d have the next 10 years to tell them in just a few months. It means to make sure everything is buttoned up so that it will be as easy as possible for your family. It means to say your goodbyes.

I lived with that diagnosis all day. Later that evening I had a biopsy, where they stuck an endoscope down my throat, through my stomach and into my intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated, but my wife, who was there, told me that when they viewed the cells under a microscope the doctors started crying because it turned out to be a very rare form of pancreatic cancer that is curable with surgery. I had the surgery and I’m fine now.

This was the closest I’ve been to facing death, and I hope it’s the closest I get for a few more decades. Having lived through it, I can now say this to you with a bit more certainty than when death was a useful but purely intellectual concept:

No one wants to die. Even people who want to go to heaven don’t want to die to get there. And yet death is the destination we all share. No one has ever escaped it. And that is as it should be, because Death is very likely the single best invention of Life. It is Life’s change agent. It clears out the old to make way for the new. Right now the new is you, but someday not too long from now, you will gradually become the old and be cleared away. Sorry to be so dramatic, but it is quite true.

Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.

When I was young, there was an amazing publication called The Whole Earth Catalog, which was one of the bibles of my generation. It was created by a fellow named Stewart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960′s, before personal computers and desktop publishing, so it was all made with typewriters, scissors, and polaroid cameras. It was sort of like Google in paperback form, 35 years before Google came along: it was idealistic, and overflowing with neat tools and great notions.

Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find yourself hitchhiking on if you were so adventurous. Beneath it were the words: “Stay Hungry. Stay Foolish.” It was their farewell message as they signed off. Stay Hungry. Stay Foolish. And I have always wished that for myself. And now, as you graduate to begin anew, I wish that for you.

Stay Hungry. Stay Foolish.

Thank you all very much.

Teamviewer: Group screen sharing tool for Ubuntu

I had been looking for a Goto Meeting / WebEx counterpart for Ubuntu machine in preparation to our project demo/presentation and found the following:

1. Skype – supports Windows, Mac, Linux machine. As of to-date, latest Skype version for Linux and Mac is 2.2 Beta and 5.3 respectively. But only Skype Premium version 5.2 for Mac users has group screen sharing feature. Rates apply.  (http://www.skype.com/intl/en-us/prices/premium/)

2. Glance – supports Windows, Mac machine only. We tried Glance Free Trial version 2.6 for Windows and plan to control my Ubuntu box remotely via VNC viewer but was a bit disappointed with the performance. Running a video from the host machine displays lag and actions performed from the host are not displayed real time. (http://www.glance.net/install/install.asp)

3. TeamViewer – FREE remote desktop access and desktop sharing that supports Windows, Mac and Linux machine. We tried TeamViewer full version 6.0 for Linux to host our project presentation to be accessed by Mac and Windows machine.

TeamViewer Installation Guide:

1. Download latest Teamviewer for Ubuntu. I got mine for Debian, Ubuntu (64-Bit) version 6.0:

http://teamviewer.com/en/download/index.aspx

2. Open .deb file with Ubuntu Software Center and Install.

3. After successful installation, you should be able to find TeamViewer 6 option under Main Menu/Internet menu.

4. Host has the option to create session for Remote support, Presentation or File Transfer. In our case we selected Presentation option because we intend to use it for project demo. Send invites to your partners/participants by giving them the auto-generated session ID and Password.

5. Partners will just need to access https://go.teamviewer.com/ and input the session ID and Password.

Indeed TeamViewer is the ideal solution for online collaboration. The best of all is it’s FREE! =D Thanks TeamViewer Team!

Error mounting HDD from my Linux machine

Encountered below error upon mounting my WD HDD in my Linux laptop:

Error mounting: mount exited with exit code 13: $MFTMirr does not match $MFT (record 0).Failed to mount ‘/dev/sdb1′: Input/output error NTFS is either inconsistent, or there is a hardware fault, or it’s a SoftRAID/FakeRAID hardware. In the first case run chkdsk /f on Windows then reboot into Windows twice. The usage of the /f parameter is very important! If the device is a SoftRAID/FakeRAID then first activate it and mount a different device under the /dev/mapper/ directory, (e.g. /dev/mapper/nvidia_eahaabcc1). Please see the ‘dmraid’ documentation for more details.

To fix, first install ntfsprogs utility by typing in the terminal:

sudo apt-get install ntfsprogs

Then type

sudo ntfsfix /dev/partitionName

In my case, sudo ntfsfix /dev/sdb1

To check if the fix was successful, these commands should be displayed:

exist@exist:~/Projects/ehealth/lib$ sudo ntfsfix /dev/sdb1
Mounting volume… FAILED
Attempting to correct errors…
Processing $MFT and $MFTMirr…
Reading $MFT… OK
Reading $MFTMirr… OK
Comparing $MFTMirr to $MFT… FAILED
Correcting differences in $MFTMirr record 0…OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition… OK
Going to empty the journal ($LogFile)… OK
NTFS volume version is 3.1.
NTFS partition /dev/sdb1 was processed successfully.

Whew! Now I was able to access my HDD =) Thanks to forums online!

Fix Asus K52J speaker sounds with Ubuntu 10.10

I don’t want to annoy my colleagues with the music I’m listening, but the sounds in my company issued laptop, ASUS K52J with OS(Ubuntu 10.10) persistently comes from the speakers even after plugging my headphones

Was able to resolve this issue by installing Linux Alsa Driver Modules

1. In your terminal, add the ppa

sudo add-apt-repository ppa:ubuntu-audio-dev/ppa

sudo apt-get update

2. Install the linux-alsa-driver-modules package

sudo apt-get install linux-alsa-driver-modules-$(uname -r)

3. Restart your machine

Thanks to this wiki page works like gem!