Reviewing the Execution Logs of Katalon Studio

While the output logs of Katalon Studio are extremely helpful, it’s not possible (as far as I can tell) to copy that output directly. The reason being, I want to run a baseline test to capture multiple data points. After a code deployment, I want to run the tests again and verify the results are the same. The logs look great in Katalon, but the raw logs contain a lot more information.

On my Mac, I can use Terminal and the Unix command, Grep, to start filtering. The cleanest file to work with is the JUnit_Report.xml file, located in the Reports directory. Inside, there are lines marked as:

[MESSAGE][PASSED]
[MESSAGE][WARNING]
[MESSAGE][FAILED]
[MESSAGE][ERROR]

The real file will contain lines that look like this:

2018-03-25 09:42:40 - [TEST_STEP][PASSED] - Statement - today = new java.util.Date(): null
2018-03-25 09:42:40 - [TEST_STEP][PASSED] - Statement - yesterday = today.previous(): null
2018-03-25 09:42:40 - [TEST_STEP][PASSED] - Statement - todayDate = today.format("MM/dd/yyyy"): null
2018-03-25 10:02:27 - [MESSAGE][PASSED] - Delayed 2 second(s)
2018-03-25 10:02:27 - [MESSAGE][WARNING] - Filter Results: 1679
2018-03-25 10:02:27 - [MESSAGE][PASSED] - Text of object 'Object Repository/Page_/Dashboard/Footer-Total Number of Users Returned' is: '1-25 of 1679 users'
2018-03-25 10:02:27 - [MESSAGE][WARNING] - Pagination Results: 1679
2018-03-25 10:02:27 - [MESSAGE][PASSED] - SUCCESS: The Filter Results Matches the Pagination Results]]></system-out>

The drawback are the [TEST_STEP][PASSED] lines. Those need to be filtered out.

At the Terminal prompt:

grep "MESSAGE" JUnit_Report.xml > firstpass.txt

This creates a text file that contains the [MESSAGE] lines.

This is good, but there will be hundreds of [MESSAGE][PASSED] lines, which really aren’t that important. That’s not a problem as those lines can be filtered out in LibreOffice using the AutoFilter.

Load the file, select AutoFilter and the option for Standard Filter. Set the filter so the lines that Do not contain the word PASSED are displayed.

libreoffice-standard-filter

Now we have our Warning, Error and Failed messages together. When the second test is completed and the results filtered in the same manner, they can be pasted into the next column to be compared line by line. Or, the EXACT function can be run to compare the two strings.

To illustrate, here is the Log Viewer from Katalon:

katalon-log-viewer

And here are the same results in LibreOffice:

libreoffice-log-viewer

If needed, the same filtering method can be used to see the Failed and Error messages. But this gives the ability to run a test, capture data, run the test again and compare the results.

Other articles of interest:

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Comments
  • Output status messages and test information by writing to the Log File Viewer in Katalon Studio (6)
    • Saranya: Good One. Could you plz share link to access all your katalon related blogs. ThankQ
    • Rodrigo Calabretta: I’m using the KeywordUtil.markFailed or KeywordUtil.markError and my test stops is being shown as Error and If I use in the @afterTestCase the testCaseContext.getTestCaseSta tus() to show the status test case...
  • Working with Dates and Date Formatting in Katalon Studio (14)
    • Don Pedro: That could be a little tricky. First, is the text/format always going to be the same? You will need to do some parsing. Separate the first but of text at 2019. After that, split the text again to get rid of Central Standard...
    • Don Pedro: It should be of the same Date type as today. You could then do some calendar trickery with Calendar /*Calendar c = Calendar.getInstance(); //c.set(Calendar.MONTH, 1) //Set the month, 0=January, 11=December println...
    • josh: I’m testing a page with a string like “Updated November 21, 2019 16:25:32 PM Central Standard Time.” How might I capture the date and time and compare it to current time to verify that the update time was within...
    • zakir hussain: today = new Date() lastWeek = (today – 1.week) can u please help me in understanding what is the return type of lastweek? and I wanted to set nextyear date how can i do it
    • ALLAN FORD: Useful !
    • Don Pedro: Not sure I understand what you are trying to do.
    • Don Pedro: formattedDate = mydate.format(“M/dd/yyy& #8221;)
  • Setting up Environment Profiles in Katalon Studio (2)
    • Don Pedro: Not quite sure I understand. The term “Execution Profile” is normally used now, but they are still GlobalVariables and are created in the same way for version 7 as previous. You make a new execution profile, then...
    • Aparna: Hello, Your documentation is excellent and has helped me understand many things. But this article above seems outdated and I need help in creating my own Global variables and all the documentation out there seems to be point to...
  • Here is my drive cluster (2)
    • Don Pedro: While I still have those drives, the drive cluster looks nothing like that now. Almost all of my external drives have had the cases removed and they have been installed into a series of MediaSonic 8 Bay Drive Enclosures. I...
    • Marin Boucher: Hi! Verry funny blog with “go to the point” on many post! I spent good time reading some of your post. Regarding this one about NAS, “many NAS” in fact, I would be curious to see a screenshot of...
  • How To Disable the Quicken Registration Prompt (27)
    • debra: didn’t work for me… my 2007 – which I am keeping on my older mac.
  • Adding entries to an open Excel spreadsheet during runtime (1)
    • IanG: Hi: Seems like nobody else has described (in detail) how Katalon can be configured to test an API by reading the first entry from a multi-row multi column XLSX spreadsheet, executing a test, writing the result to another column (or...