Marking tests as Passed, Failed or in Error using MarkFailed, MarkPassed, MarkError in Katalon Studio

We previously looked at the idea of writing information to the log files. This included the output of variable values and status messages. But, what if you need to stop the test and mark it as failed because the data you needed wasn’t there, or the information you did read indicates a more serious problem? Katalon Studio will mark a test as failed if the command fails to execute, but there is a way to fail the test because of other errors. Again, it takes a little digging to find the right command, but they certainly exist.

To start, an import statement is needed at the top of the project:

import com.kms.katalon.core.util.KeywordUtil

Once that’s in place, a test can be flagged in several ways using one of these commands:

KeywordUtil.markPassed
KeywordUtil.markFailed
KeywordUtil.markError
KeywordUtil.markWarning

* https://api-docs.katalon.com/studio/v4.6.0.2/api/com/kms/katalon/core/util/KeywordUtil.html

With these KeywordUtil commands, you have a great deal of flexibility in controlling the execution of your tests. For this small snippet, the two values I’m working with are written to the log file. I then use KeywordUtil to mark the test as passed or failed if those two values match or not.

log.logWarning('The All filter displays: ' + filterAllUsers)
log.logWarning('The number of users for Pagination displays: ' + footerAllUsers)
if(filterAllUsers==footerAllUsers){
 KeywordUtil.markPassed('SUCCESS: The Filter Results Matches the Pagination Results')
} else {
 KeywordUtil.markFailed('ERROR: The Filter Results Does NOT Match the Pagination Results')
}

The test itself will pass since there is no error in the code or in finding the elements on the page. However, if the values I’m comparing aren’t equal, I need to fail the test so I can look at it manually and determine the problem.

As another example, I need to check one of our sales dashboards. Due to some data inconsistencies during import, it’s possible for the month to display twice on the page, as in January, January, February, February.

Verifying these duplicate values won’t get the test to fail, the code executes correctly. But, now I can force it to error when I evaluate a duplicated month.

//Count the number of months in the table. If there are more than 12, there's an error with duplication
log.logWarning('Number of Months displayed in the Table: ' + TotalRowCount.size())
if (TotalRowCount.size() > 12){
 log.logWarning('There are more than 12 months. This is an error.')
 KeywordUtil.markError('ERROR: There are more than 12 months. This is an error.')
}

//If the above passes, such as an onboarded user, check that the text of the first month isn't the same as the second
//Compare the first two months, display the first three for visual comparison
monthName1 = driver.findElement(By.xpath("//*[@id='byMonth']/div/table/tbody/tr[1]/td[1]")).getText();
log.logWarning('Month:=' + monthName1)
monthName2 = driver.findElement(By.xpath("//*[@id='byMonth']/div/table/tbody/tr[2]/td[1]")).getText();
log.logWarning('Month:=' + monthName2)
monthName3 = driver.findElement(By.xpath("//*[@id='byMonth']/div/table/tbody/tr[3]/td[1]")).getText();
log.logWarning('Month:=' + monthName3)
if (monthName1==monthName2){
 log.logWarning('The two months sampled have the same text. The months appear to be duplicated.')
 KeywordUtil.markError('ERROR: The two months sampled have the same text. The months appear to be duplicated.') 
}

Again, I’m using both log.LogWarning and KeywordUtil.markError to output values and then mark the test as in error.

A test can be passed or failed for almost any reason you can control. As another example, here is a snippet of code that indicates whether a Contact was created successfully if the popup dialog indicates there was a success.

elementPresent=WebUI.waitForAlert(20)
if (elementPresent==true) {
 alertText = WebUI.getAlertText()
 log.logWarning('The title of the alert is:=' + alertText)
 if (alertText=='Successfully Created Contact'){
 KeywordUtil.markPassed('SUCCESS: Successfully Created the Prospect')
 WebUI.acceptAlert()
 } else {
 WebUI.acceptAlert()
 KeywordUtil.markFailed('ERROR: There was an error creating the Prospect')
 }
}

After the contact is created, the code waits for an alert to display. For that alert, read the text. If the text is “Successfully Created Contact” then obviously all is well. If it reads as something else, then a problem has occurred and the test will be marked as Failed so I can review it later.

It’s also possible to brute force end the execution of the test using the throw command. The following will stop the test in it’s tracks and log an error:

throw new AssertionError('ERROR: There was an error creating the Prospect')

As noted before, the count of Passed, Failed and Error tests is displayed on the Log Viewer.

Katalon offers a huge amount of flexibility with the inclusion of KeywordUtil.markFailed and KeywordUtil.markError. It’s now possible to indicate there is a problem even when all the commands execute successfully. Or, you can exit out gracefully if you know the information you just read would cause the rest of the test to fail. An ideal case would be reading the value $0, right before you perform a calculation.

Other articles of interest:

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Comments
  • Working with Dates and Date Formatting in Katalon Studio (15)
    • Sangeethaa: How to get previous day’s date(Yesterday’s date)? Actually I was using today.previous() before,it was working fine till today morning.Now its not retrieving right date. Can anyone please guide me on this.
    • Don Pedro: That could be a little tricky. First, is the text/format always going to be the same? You will need to do some parsing. Separate the first but of text at 2019. After that, split the text again to get rid of Central Standard...
    • Don Pedro: It should be of the same Date type as today. You could then do some calendar trickery with Calendar /*Calendar c = Calendar.getInstance(); //c.set(Calendar.MONTH, 1) //Set the month, 0=January, 11=December println...
    • josh: I’m testing a page with a string like “Updated November 21, 2019 16:25:32 PM Central Standard Time.” How might I capture the date and time and compare it to current time to verify that the update time was within...
    • zakir hussain: today = new Date() lastWeek = (today – 1.week) can u please help me in understanding what is the return type of lastweek? and I wanted to set nextyear date how can i do it
    • ALLAN FORD: Useful !
    • Don Pedro: Not sure I understand what you are trying to do.
  • Output status messages and test information by writing to the Log File Viewer in Katalon Studio (6)
    • Saranya: Good One. Could you plz share link to access all your katalon related blogs. ThankQ
    • Rodrigo Calabretta: I’m using the KeywordUtil.markFailed or KeywordUtil.markError and my test stops is being shown as Error and If I use in the @afterTestCase the testCaseContext.getTestCaseSta tus() to show the status test case...
  • Setting up Environment Profiles in Katalon Studio (2)
    • Don Pedro: Not quite sure I understand. The term “Execution Profile” is normally used now, but they are still GlobalVariables and are created in the same way for version 7 as previous. You make a new execution profile, then...
    • Aparna: Hello, Your documentation is excellent and has helped me understand many things. But this article above seems outdated and I need help in creating my own Global variables and all the documentation out there seems to be point to...
  • Here is my drive cluster (2)
    • Don Pedro: While I still have those drives, the drive cluster looks nothing like that now. Almost all of my external drives have had the cases removed and they have been installed into a series of MediaSonic 8 Bay Drive Enclosures. I...
    • Marin Boucher: Hi! Verry funny blog with “go to the point” on many post! I spent good time reading some of your post. Regarding this one about NAS, “many NAS” in fact, I would be curious to see a screenshot of...
  • How To Disable the Quicken Registration Prompt (27)
    • debra: didn’t work for me… my 2007 – which I am keeping on my older mac.
  • Adding entries to an open Excel spreadsheet during runtime (1)
    • IanG: Hi: Seems like nobody else has described (in detail) how Katalon can be configured to test an API by reading the first entry from a multi-row multi column XLSX spreadsheet, executing a test, writing the result to another column (or...