Katalon regression project is complete

Good news! My QA automation project is now complete. Well, complete in the sense that I have a full test suite to run against QA and Production. No code project is ever really complete and there is plenty more for me to work on. But, at this point, I have a fully functional project with 30 tests that cover a majority of the functionality.

I recently finished the regression suite with all the tests running together. Small adjustments were needed due to some small timing issue, and a few leftover hardcoded values I missed on my previous pass.

A few initialization scripts were also written to setup and retrieve data before the test fully got underway. The suite also did it's trial by fire, by being used in a Production smoke test for our last deployment.

I'm obviously not going to let it run completely unsupervised, but it did very well. It checked all the data points it was supposed to and verified all the elements I was concerned with. By that measure it was extremely successful.

The next step is to keep writing and refining tests. Currently, there are no "create" tests, meaning nothing new is added to the system using this code. That will be changed for a few items as we moved forward. The next big step will be to make a complete "transaction" and verify all the information on that page.

There are several smaller tests that can be added, which will be quick to write since all the framework is in place and most of the functionality is a copy and paste of work already completed.

Like previous tests I've done, I will be getting with our product owners to see what other kinds of tests would be worthwhile. What bugs or issues are we seeing that could be detected with code? Is there something customer repeatedly do or an area that needs additional coverage/attention?

At this point, the project is working well and has uncovered a few bugs and logic flaws. When you step through the site one page at a time, one item at a time, and dig into options you normally gloss over, some issues are bound to come to light. Doing this sort of work is a great way to get reacquainted with all aspects of a site.

Other articles of interest:

How to get the URL of an image/icon with Katalon

For a test I'm working on, the status of an item in a table was shown using an icon. It is a yellow triangle for warning and a green checkmark for passed. For my test, I need to know which icon is displayed and act accordingly. In order to check that, I ended up reading the URL of the icon and checking which .PNG file is displayed. Like other elements on the page, the image needs to be defined as an object.

To define the image, I created an object with this reference:


It's sort of an odd reference since the icon is in a row within a table. But in essence, the /img lets us know we are defining and img object.

To get the text or URL of the object, we need to use the GetAttribute command:

agreementSigned=WebUI.getAttribute(findTestObject('Business Locations/img-Signed Agreement', [('row') : counter]), 'src')

For the above we have the .getAttribute which looks at the definition of our img object. We define the attribute as src since we want the image source. This will return either:

https://url/check_16.png or https://url/warning16.png

We can then use contains to see which icon we have:

if (agreementSigned.contains("check_16.png")!=true){

I haven't had to use this before, but getting the answer and putting it together took longer than I would have liked, so hopefully this helps someone out.

Other articles of interest:

Automated Functional Testing

With a pretty solid framework in place, I'm starting to use my automation code for functional testing. I've been able to adapt completed test to accomplish specific goals.

For example, I recently made a new test to add new products to our system. Each product has several options that can be configured. I was able to make a test to programmatically add several hundreds new items, in multiple configurations, and get the test done in a single day.

Using what I already had, I was able to copy and paste 50% of the code as my base. I then used the standard commands already used to populate the form the way I needed.

From my testing, it took me about 30 seconds to populate the form and click the different options. The code does it in 5 seconds. That is also 30 seconds at my best. If I were to have to sit there for 4 hours straight, not only would I stare blankly into the abyss, I would make dozens of mistakes and it would take longer and longer to complete.

I let it run for about 4 hours during the test period and made hundreds of new products, each one slightly different in name and options selected. By the time is was finished, just about every combination possible had been created. That's a pretty good test run.

Other articles of interest:

The automation project continues to blossom

My automation project is coming along very nicely. While not a very good gauge of progress, lines of code is how I track what I'm doing. At this point, I have crossed over 3,000 lines of code mark.

I'm not writing gibberish, as that doesn't help me. My goal is to write tests as quickly as possible, not necessarily as efficiently as possible. I'm not a dedicated automation engineer, so if I can't write, test, debug and complete the test in a single day, I'm painting myself into a corner.

I take a piece of functionality and break it down. I have 4-5 hours to complete whatever it is I'm attempting. If it's not ready to be committed by the end of the day, I need to rethink what I'm doing. Either my task needs to be broken down further, or I'm trying to do something that is more complicated than it needs to be. I don't need to be fancy.

Yes, I may repeat blocks of code that could be more efficient. I may copy and paste several things and only change 1 or 2 objects. That refactoring will come. But, when I start, I am following the quickest path to resolution.

If it takes me two hours to put a test together by copying and pasting the same thing a dozen times, I'm fine with that. I'm not being graded by how my code looks.

There is no point in spending 10-15 hours putting a test together that only does an hours worth of work. That's not efficient from a time perspective.

If I spend 2 hours putting something together, then run it for 4 hours, and make several hundred passes, that's worth the time. I can then go back with the time I've saved and make the test more efficient.

As I've said, I go through each test several times, making adjustments and improving it. But, at each state, it's still full functional and ready to be called into service to get the job done.

It's working and saving me time, that's all I care about.

Also, once we get to the usual lulls that come along, I turn around and do the opposite with the project. I mark my progress by how much code I can take out.

Other articles of interest:

QA Regression Suite coming together

My automation project is progressing faster than I expected. I added the code to check for results, added code to count rows, and removed all the hard coded values I put in place.

The QA Execution Profile is now in place with the correct data values for that environment. I even swapped some of the String values for Lists so multiple values could be checked in a single pass.

I cleaned up the navigation code so the tests can either click the links they need, or they navigate to the correct location then call each other in the proper order.

I have just put together the QA Regression test suite and have been testing it against the site. It contains 21 tests, chained together, that run from start to finish.

With everything running together, I made a few adjustments to my data values, picked some better customers to work with, added a couple of checks to make sure objects were visible before starting, and added logging for a couple of events.

It’s not a comprehensive site validation suite, but it does perform one hundred actions or so, and completes in around 7 minutes. For me to perform the same actions is closer to 40 minutes.

At this point, I need to try different kinds of users and see the how the tests react. Do they fail gracefully? Do customers have data I didn’t anticipate? Are they missing something I need?

For next week, I will try this for a few dozen customers we have and see things work. Once that is completed, I will be ready to duplicate this Test Suite as the Prod version and go through the same process of trying it against the real Prod site.

I am very much on target to use this as for the after deployment smoke test we have scheduled at the end of the month.

Other articles of interest:

Recent Comments

  • How To Disable the Quicken Registration Prompt (32)
    • Mike: Hi, for anyone still trying to run older versions of Quicken, mine is 2004, I hope this helps. I managed to get it running on my old laptop by adding the three missing dll files, when I needed to upgrade I thought I would be able...
    • Juani: Hi, I have a registered Quicken 2016 and now my files are being held hostage by Intuit, cannot log in with my ID unless I upgrade therefore buy a subscription, I need to access my files, please HELP. Thanks
    • David: A BIG Thank you! I regularly reinstall Windows OS’s using different hardware configs. In short, Quicken 2011 (it’s March of 2020 as I write this) has been a program I’ve been very happy with and continue to use....
    • Karen L: Today I rang Reckon Australia again, got a different guy and he talked me through the process of getting me a new product key to enter since I already had the licence and it had been extended. Reckon have only extended it by 1...
    • Karen L: I have Australian Quicken 2008 on Windows 8.1. It keeps asking me to renew my licence which Reckon did for me however the activation won’t work. I tried this to disable the registration prompt however it did not stop the...
  • Enter dates into a date picker for Chrome and Firefox (1)
    • Prasanna: Hello, How to enter a previous month FIRST day and LAST day from the system date. Thanks Prasanna
  • Building Relationships with Developers (1)
    • Carlos Herrera: Ah yes. I have a team of software developers and it is genuinely so hard to keep them in the loop during projects. Though the more we work the better the communication becomes and we go from a trail and error process to a...
  • Integrating DevonAgent Pro with Alfred (1)
    • J. Garr: Beautiful, sweet, and simple. I love it; thanks for posting this solution.
  • Round and Round with the For..Next in Katalon Studio (1)
    • Sweet Ophaline Labador: Hello can you help me how to loop this scenario. Checking the elements is just the same. I want to check that the following function is available in Track Shipment – clicking on this can...
  • Parsing Strings in Katalon – Split, Substring and Readlines (6)
    • Don Pedro: String tempText=”Date Month January. Revenue $1,355,721.00″ println(tempText.split(“ \$”)[1]) You need to escape the $ with \$ The [1] on the split captures characters on the right of the split [0]...
    • Prasad: Hi, I need to get the substring from the below string ‘Date Month January. Revenue $1,355,721.00’ I want to extract text after $ value. Please help Thank you
  • Working with Dates and Date Formatting in Katalon Studio (15)
    • Sangeethaa: How to get previous day’s date(Yesterday’s date)? Actually I was using today.previous() before,it was working fine till today morning.Now its not retrieving right date. Can anyone please guide me on this.
    • Don Pedro: That could be a little tricky. First, is the text/format always going to be the same? You will need to do some parsing. Separate the first but of text at 2019. After that, split the text again to get rid of Central Standard...
  • Output status messages and test information by writing to the Log File Viewer in Katalon Studio (6)
    • Saranya: Good One. Could you plz share link to access all your katalon related blogs. ThankQ
    • Rodrigo Calabretta: I’m using the KeywordUtil.markFailed or KeywordUtil.markError and my test stops is being shown as Error and If I use in the @afterTestCase the testCaseContext.getTestCaseSta tus() to show the status test case...