Don Pedro

Connecting Katalon Studio into XLS and CSV files

Since reading from external source files has become an important part of how we get data, I wanted to put together some simple examples to show reading data from a CSV file, an XLS file and from a Database source. All three are very similar and since Katalon handles most of the work, it's easy to grab data from a file with just a few lines of code.

We start by using the TestData library and reference the "Test Data File" from the "Data Files" object in the project. That is done with:
TestData csvData = findTestData("Data Files/Data Source Name")

Once we have that, we can use some of the built in Katalon keywords to get the number of rows in the file and the column names.
int rowCount=csvData.getRowNumbers()
List columnNamesList=csvData.getColumnNames()

Once we have that information we know how many rows we can read and how many columns. We then use getValue to read our piece of information.
temp1=csvData.getValue(columnName, rowNumber)

When working with a database, there is one small addition. We can use getAllData to retrieve all the items that have been returned from the query. This isn't a necessary step since we can still get the number of rows and parse through the data. However, if the entire block of data is needed, getAllData can be used.
List resultSet=dbData.getAllData()

In such a case, your data will be returned in this format:
[[04GA, 61], [04OC, 93], [02OC, 334]]

For the above we have the SKU followed by the quantity.

Again, Katalon provides a very simple way of connecting into different kids of data sources and retrieving information. This has certainly improved my Test Cases and opened up a variety of test possibilities.

Read data from CSV file

TestData csvData = findTestData("Data Files/csvDataSource")

int rowCount=csvData.getRowNumbers()
println rowCount
int rowNumber=1
List columnNamesList=csvData.getColumnNames()

columnName=columnNamesList.indexOf("columnName")+1
println columnName
temp1=csvData.getValue(columnName, rowNumber)
println temp1
Read data from XLS file

TestData xlsData = findTestData("Data Files/Inventory Items")

int rowCount=xlsData.getRowNumbers()
println rowCount
int rowNumber=1
List columnNamesList=xlsData.getColumnNames()

columnName=columnNamesList.indexOf("Item Name")+1
println columnName
temp1=xlsData.getValue(columnName, rowNumber)
println temp1
Read data from Database

TestData dbData = findTestData("Data Files/PostgresDB")
numOfResults=dbData.getRowNumbers()
List columnNamesList=dbData.getColumnNames()
println columnNamesList
List resultSet=dbData.getAllData()
for (loop = 0; loop <=(numOfResults-1); loop++) {
    println(resultSet[loop])
}

temp1=dbData.getValue(2, 1)
println temp1
temp2=dbData.getValue(3, 1)
println temp2
columnName=columnNamesList.indexOf("column name")
println "column Position:" + columnName

Other articles of interest:

Udemy courses in Java and SQL

Over the past two years I have taken a lot of really good training courses at Udemy. It's a great way to get instructor led training on a wide variety of technology topics. From that, there are two courses that rise to the top because they are very well done and provide a wealth of information to the QA engineer.

The first is a programming course. Since Java is the language of automation, I highly recommend:

Java Programming Masterclass for Software Developers by Tim Buchalka

You know you're in for a lot of information when the course is a staggering 80 hours of content. It covers the basics of Java, how to get it installed, picking an IDE, a quick overview of data types, and how to get your first programming running.

Tim then gets into the meatier topics of classes, objects, inheritance, constructors and encapsulation. And for each topic there are plenty of code examples, exercises and demos. There also lots of conversations about programming methodology to hold things together.

While not an automation course, it provides a wealth of information to develop and improve your code. I've learned a huge amount from this course and I don't consider myself to be a developer.

The second course on the list is:

The Complete SQL Bootcamp by Jose Portilla

To me, SQL is an umbrella term for many things. There is the language, the database itself, database theory, architecting, maintenance, and development (stored procedures).

That's not where I need to be. I'm not a SQL engineer, but writing a query is a huge benefit to the QA engineer. We can validate site information and provide dynamic data for Katalon scripts. This course focuses on building those kinds of queries.

It starts off with the basics of SQL, the SELECT statement then moves into COUNT, JOIN, ORDER BY, LIKE, WHERE, BETWEEN and GROUP BY.

For your own practice, it goes over the basics of installing and creating a database, setting up data types, and gives a solid foundation for starting with SQL – the language.

I am by no means an expert, but the topic is no longer a total mystery to me. As noted, I've been able to connect Katalon into our database and retrieve data. While I'm not the fastest at putting a query together, I have a far better understanding of what I'm doing.

As a matter of fact, I plan to retake this course during this slow period of the holidays as a refresher and to dig deeper into topics now that I have a better understanding.

Other articles of interest:

Why use a Mac for development?

There are multiple reasons why I develop and test on the Mac. Primarily, it's the platform my company has chosen. But beyond that, over the last two years I have replaced almost all of my Windows machines with Mac computers. I have a Mac Pro at work and at home, along with a Mac mini, a pair of iMacs, and a few combinations of MacBooks. Not to mention my collection of iPads going back to the original model, and a couple of iPod Touch devices.

Well before that, I got my start with the Apple II, a machine I still hold in very high regard. By the time college came around, I was working on PC clones with only occasional stints on a Mac. Then it became all Windows based machines.

I didn't really come back to Apple until the original iPad came out. To me, that device was magical. It was so utterly brilliant in both form and function.

I contemplated switching to the Mac several years ago, but I've always been with Windows centric companies. Now that I'm with a company that embraces Mac, open source, and web technology, I'm not specifically tied to an OS.

After a time, I decided to buy my own development machine. I could have brought a machine from home, a Windows machine with 8 cores with 32GB of ram, which is pretty good. To keep in step with my renewed interest in the Mac, I saw I could get a Mac Pro with 2 physical processors for 12 cores and 128GB of ram. It's an older machine, but those are still impressive specs.

And with that machine I have completed tasks and projects I don't think I could have done on another machine.

I'm able to run multiple instances of Katalon for performance testing, even though that's not quite how it should be done. I can push JMeter to the point of saturating our network. I can have dozens of tools and applications open across multiple screens without worrying about system resources. I have helper applets galore without seeing my system slow to a crawl. I have instances of Firefox, Chrome and Safari running without issue. I've even had multiple VirtualBox machines running for browser testing.

It's not just the machine, the OS has played a very important part in the development work I've taken on over the last two years.

I have found macOS to be incredibly stable. I think the last time I needed to reboot my machine was 6 months ago. I am not a fan of the Windows 8/10 UI and find the macOS style far more appealing and unobtrusive.

Installing and removing apps is incredibly simple. There is no registry or files scattered all over the place or the need to reboot.

Further, the Mac licensing is far more generous so I have the same apps at home as I do at work.

I also feel there is better software that fits my needs. There are simple things like window managers, text expanders, and clipboard editors. But there's also far more calendaring, task management, project management, document management, note taking, markdown and text editing packages available for the Mac than Windows.

It also feels the Mac is more developer centric. Coding for the .NET framework is done on Windows, but it's a big world outside of Visual Studio. When it comes to working with Java, Groovy, Python, you will find more answers that don't involve Windows.

And there is no shortage of development tools for the Mac. Not just IDEs. There are multiple options for storing code snippets, for parsing text, for managing the clipboard, for storing notes and data. As the saying goes "There's an app for that."

Finally, there is the power of Unix under the hood. Terminal offers the hundreds of tools native to Unix as well as shell scripting which can be amazing. The power of grep compels you!

I've had a lot of success with Windows in the past. I'm now taking that further with the Mac and enjoying it quite a bit more.

Other articles of interest:

My Latest TypeIt4Me Stats

I’ve mentioned on several occasions that I am a big proponent of text expander tools. With my personal choice being TypeIt4Me.

A text expander can save huge amounts of time when it comes to writing code, writing documentation, and all sorts of general computer tasks.

Just to show it off, here are the latest stats of TypeIt4Me from my main machine at work.

typeit4me

Other articles of interest:

Excel is not a documentation or testing tool

I have seen this coming up in multiple places and wanted to make my own comment. While Excel and spreadsheets in general are fine tools, they are NOT documentation or testing tools.

I know it's become the norm to create test cases and even test plans and publish them as spreadsheets so they are easily shared. But to be honest, it pains me to no end.

Excel is fine for making a basic checklist, or listing ticket numbers associated with a project and handing it over to the client to or to someone in the business so they can check the progress at a glance, but using it to store API data, SQL queries, write elaborate steps that someone should follow to perform an action, color coding dozens of columns, creating dozens of sheets filled to the gills with test permutations, or using it to backlog tickets, sends me into a terrible frenzy.

Simply put, there are far better tools out there for handling that kind of data, they are called Task Managers. If you need more than that, look for a Project Management tool.

I use Excel or rather LibreOffice to create a summary list of what I am working on for the client. But it's not something I actually work from. I use 2Do to create my own task list and keep track of my progress. I use DevonThink to track requirements documents, credentials and page links for my own personal benefit. I use Jira and Confluence for bug tracking and sharing test information.

I have seen too many cases where everything is stuffed into Excel because that's the tool someone has available and the only tool they know how to use. There are dozens of tabs filled with color coded data, lines going in all directions, screenshots plastered all over the place, and all sorts of bug descriptions.

To this I say no. It's a terrible idea. Bugs should be tracked in the correct bug tracking tool, whether that is Jira or something else. Screenshots need to be attached to the ticket or the requirements document. Simply put, the correct data should be listed in the correct place, not stuffed into an Excel spreadsheet someone made.

It also leads to the problem of data hoarding. It's far to easy to store everything inside a spreadsheet in a way that only makes sense to you, that is stored in a folder on your machine, that isn't shared and takes multiple steps to make available to someone else.

I'm a huge fan of Excel. I used to support it. It can do marvelous things. But just because it can, doesn't mean it should.

I have run into far too many spreadsheets that are stuffed to the gills with data that isn't listed anywhere else. The bugs aren't being tracked correctly. The testing data is locked away where other team members can't get to it. And in a recent case, so much crap was hoarded in spreadsheets, that not only are there dozens of them that don't make any sense, but dozens of them weren't actually shared after said person left. We are literally missing documentation because it was more important to be in control and use Google Drive than to share test data in a meaningful way with the rest of the team.

Yes, some of that is a personnel problem, but in my opinion, there is too much reliance and emphasis on using Excel as a testing tool. It can be used for a lot of things, and it has a lot of convenience, but I have a hard time believing spreadsheets are the best way to share testing information with team members.

Other articles of interest:

Recent Comments

  • Working with Dates and Date Formatting in Katalon Studio (14)
    • Don Pedro: That could be a little tricky. First, is the text/format always going to be the same? You will need to do some parsing. Separate the first but of text at 2019. After that, split the text again to get rid of Central Standard...
    • Don Pedro: It should be of the same Date type as today. You could then do some calendar trickery with Calendar /*Calendar c = Calendar.getInstance(); //c.set(Calendar.MONTH, 1) //Set the month, 0=January, 11=December println...
    • josh: I’m testing a page with a string like “Updated November 21, 2019 16:25:32 PM Central Standard Time.” How might I capture the date and time and compare it to current time to verify that the update time was within...
    • zakir hussain: today = new Date() lastWeek = (today – 1.week) can u please help me in understanding what is the return type of lastweek? and I wanted to set nextyear date how can i do it
    • ALLAN FORD: Useful !
    • Don Pedro: Not sure I understand what you are trying to do.
    • Don Pedro: formattedDate = mydate.format(“M/dd/yyy& #8221;)
  • Setting up Environment Profiles in Katalon Studio (2)
    • Don Pedro: Not quite sure I understand. The term “Execution Profile” is normally used now, but they are still GlobalVariables and are created in the same way for version 7 as previous. You make a new execution profile, then...
    • Aparna: Hello, Your documentation is excellent and has helped me understand many things. But this article above seems outdated and I need help in creating my own Global variables and all the documentation out there seems to be point to...
  • Here is my drive cluster (2)
    • Don Pedro: While I still have those drives, the drive cluster looks nothing like that now. Almost all of my external drives have had the cases removed and they have been installed into a series of MediaSonic 8 Bay Drive Enclosures. I...
    • Marin Boucher: Hi! Verry funny blog with “go to the point” on many post! I spent good time reading some of your post. Regarding this one about NAS, “many NAS” in fact, I would be curious to see a screenshot of...
  • How To Disable the Quicken Registration Prompt (27)
    • debra: didn’t work for me… my 2007 – which I am keeping on my older mac.
  • Adding entries to an open Excel spreadsheet during runtime (1)
    • IanG: Hi: Seems like nobody else has described (in detail) how Katalon can be configured to test an API by reading the first entry from a multi-row multi column XLSX spreadsheet, executing a test, writing the result to another column (or...
  • Output status messages and test information by writing to the Log File Viewer in Katalon Studio (4)
    • Don Pedro: That is correct. You will need to add KeywordUtil.markFailed to change the Failures count. Other commands in the library are: KeywordUtil.markPassed KeywordUtil.markFailed KeywordUtil.markError KeywordUtil.markWarning
    • Nadim: This is really helpful … the only problem when log.logFailed executed it doesn’t update the results Failures count … still show Failures: 0