Don Pedro

How I Use the Shell

I am by no means a master at programming the shell. I'm not even sure my skills would be considered novice. However, even with just a few commands, the Terminal/Unix Shell of the Mac can be an incredibly powerful tool. Things get more exciting when you couple it with Keyboard Maestro.

A few of my most common shell commands include grep, split,find, mv,cp cat, wc, head, tail and rsync. Even with just these few you can perform some very clever tricks and save yourself a ton of time.

Grep is used to find text within a text file. That may not sound like much until you try it with a .CSV file that's 1.5GB in size. Go ahead and look for a customer, SKU, or phone number in that mess. With grep it's light work and results can be piped off into a much smaller file you can open.

In fact, grep was my re-introduction to the Shell when using the Mac. The other member of the QA team was trying to open a gigantic file in Excel, and failing miserably. It had millions of lines, far more than Excel could parse. The better choice would be a database, but all he needed was to confirm if an entry existed. His goal was to open it and use AutoFilter.

After cringing at his plan, I took the file and the items he wanted to find and exported each result to a separate file. I sent those 2-5k files back to him a few minutes later and the problem was solved.

Split does what the name implies. It splits a file into smaller parts. If you actually need to open that 1.5GB CSV file, you can use split to make a dozen small files that can be opened. That has come in handy multiple times. If grep can't save you, split might get you out of a jam.

Find searches for files that match a certain criteria. I use this to find files of a certain size or certain type. There are multiple options, so you'll need to do some experimenting, but it's incredibly useful.

mv and cp are for moving and copying files. The power comes from using wildcards. I use this to move tons of files around quickly without having to select each and every one of them. Sure, there are tools to handle that, but there's no need to buy additional software when it's built it. mv *.doc is far faster than trying to click them all by hand.

rsync is a fantastic tool for backing up files. I use this to back up my DevonThink Office Pro files and documents to an external drive each week. The reason for doing it this way is that I can then copy those files to my home machine and not overwrite anything. I want everything from the work machine on the home machine, but not the other way around. This keep my home Mac up to date without losing any differences between the two. It also removes the backup once the process is complete so I don't run out of space.

Cat is used to view and concatenate files together. Easily put together a list of results.

Wc is "word count" and is great for checking the number of lines in a file.

Head and tail show the first X number or last X number of lines in a file. Excellent way to see headers, or make sure you have a complete file without having to open it.

While these commands are extremely beneficial, you can make them even more powerful and easier to use by combining them with Keyboard Maestro.

For example, I have the above sync scheduled to occur on Friday. The timing and action are handled by Keyboard Maestro. I get a prompt to connect the drive, the sync occurs, and everything is handled effortlessly. It took a bit of searching to get the right command and switches, but now it's set up and I don't have to worry about it. Keyboard Maestro adds the functionality of timing, prompts, and execution.

The same is true for mv and cp. I created a small UI using Keyboard Maestro to select common folders, set up wildcards, and pass the command over for execution. I don't have to remember the syntax and get everything in the right order. Again, I set it up once, then use it dozens of times to organize files and folders.

The Shell is it's own programming language as well. So commands can be chained together, variables set and passed, output from one used as the input for another. That is definitely a skill, but absolutely possible and incredibly powerful.

There are an amazing number of applets built into Unix. It may take some digging to find the right one, and a bit of experimentation to get the right options, but they are very well documented, along with hundreds of examples on various sites and forums. There are commands in the Shell that can easily save you having to buy an application.

Other articles of interest:

Affinity offers big discounts and extended trials on Photo, Designer and Publisher

Affinity is back with another big discount and extended trial versions to help out those working remotely or who need to finish their projects. In their own words:

A message from the Affinity team
As a way to lend support to the creative community during these difficult times, we’re once again offering a 90-day free trial of the Mac and Windows versions of the whole Affinity suite, for anyone who wants to use them (even those who have previously completed a free trial). We’re also bringing back the 50% discount for those who would prefer to buy and keep the apps, including our iPad versions.

This is a fabulous discount from a great company.


Other articles of interest:

Improve Your Writing With PaperEdit

When it comes to revising your first draft, whether it be blog article or book chapter, the task can be daunting. Of course, there are tools like spellcheck and grammar check in macOS that are great, but what about other problems like, weak phrases, long sentences, passive voice, hyperbole, and other wording to clean up.

While it’s no substitute for manually proofreading, tools like PaperEdit can help you focus your editing efforts on these specific areas.

PaperEdit allows you to load or paste text, then highlights sentences that could be improved. As mentioned it will highlight passive voice, long sentences, recycled starting words, and repeated words.

At a glance you can see where to focus. You can then cycle through each highlighted passage to see if you agree.

I find this a great first step in the revision process. If there’s a rainbow of color, it’s time to make some serious edits. That’s not a bad thing because I know where to look. I can focus on those changes before tackling the document as a whole.

After those corrections, it’s time to look at the whole document. Even if PaperEdit gives the all clear, manual proofreading is still part of the process.

Some people frown on proofing tools. I disagree. Relying on them as your only revision practice is a bad idea, but there’s nothing wrong with getting the low hanging fruit. I like the highlights, and the ability to see problematic areas at a glance.

PaperEdit is a recent addition, but I’ve gone over several articles and like the results. Plus, it’s always good to start with the basics because fixing those problems has a noticeable domino effect. It makes you rethink what you’re trying to convey.

A downloadable demo is available, although you need to scroll through the Blog section to find it. The demo isn’t the latest, but easily shows off the features. I found it valuable enough to head to the App Store for the full version.

If you want a solid first step in revision, PaperEdit is a good choice. It can also open Scrivener projects, so you don’t have to copy and paste.

You can find more about PaperEdit at Novellus Software.

Other articles of interest:

How I Use Scrivener for Journaling

Despite the "dear diary" image the word journaling may conjure, it's an extremely beneficial tool and process. It's something I have been doing with Scrivener for years.

Travel writers keep a journal of the places they visit, sites they see, people they meet. Scientists keep a journal a of their experiments, the results, and discoveries. Why not do the same thing for the journey of life?

The "journal" can take many forms and isn't always written.

Some people meditate to clear their minds or to focus on specific ideas.
Some use yoga for the same purpose.
Others listen to music and step away from tasks and chores for an hour. This allows their mind to explore and wander to new ideas.
Others take time to write gratitude and affirmations.
Some keep track of goals, activities, or events.

For me the journal and the process of writing in it serves several purposes.

At times it's a way to warm up for article writing.
If I need to focus on a topic, I write down extraneous ideas so I don't forget them.
It's a way to brainstorm. To write down everything that comes to mind about a topic to see how they fit together.
Other times it's a chronicle of the next steps I need to take in a project.
On most days it's a reflection of events, feelings, good ideas, opportunities to do better, and a general clearing house.

Because of the way Scrivener works, it's the perfect vehicle for this type of writing. As mentioned before, I have a folder for each month of the year. In each folder is a file for each day of the month I write something. And within that file is anything and everything I think is important at the time.

It's a free flow of ideas, thoughts, feelings, projects, and anything that springs to mind. Any idea that comes to mind is jotted down. There is no spell checking, no grammar checking, no revising, no deleting, no stopping to go "That's a stupid idea." The goal is to write down as much as possible as fast as possible.

Going through this process on a daily basis for several years has taught me many things. Using this "quiet time" I'm able to write down and chain together dozens of ideas in ways that still surprises me. Ideas flow together without me realizing. I see common threads and themes.

For example, I keep writing about Idea X, so maybe I should act on that. Or after writing about a certain topic, another one springs to mind and they are related in a way that didn't seem obvious to me before. I was able to add links to the chain in a natural and organic way.

If the same frustration keep rearing it's head, I need to figure out how to deal with it. What's at the root of that frustration? What's at the root of that action that generates the reaction?

When thinking about an article topic, many new ideas spring out. In many cases what appeared to be a somewhat bland topic, reveals itself to be far more complex with lots of avenues to explore.

What I have learned is that when you allow yourself to express whatever ideas you have without trying to justify them, all sorts of things take root.

For example, what's wrong with writing down anything and everything related to a business idea you have. You're not trying to convince anyone. You're not asking for money. You're writing down how awesome the idea is, your expectations, where you believe it will lead, how things will turn out, how you feel about those outcomes.

Same goes for a story idea. You're not trying to sell the idea. You're not trying to publish anything. You're mapping out how great those characters are and how fantastic this adventure is going to be. Let the stream of consciousness whisk you along and see where it goes.

Additionally, no one has to ever hear or criticize these ideas. Others aren't privy to hear what you're grateful for unless that's your choice. No one needs to see your life affirmations unless you share them.

Two years ago, I found a list of "writing prompts." Fifty two in fact, one for each week of the year. The goal was to take the prompt and write whatever came to mind. There was no "right" nor did you need to stick to the topic. It was an exercise to get you thinking and writing. I put those in Scrivener, each prompt in a document, and filled them out on Sunday. Some were serious, some were for fun. But, by sitting down and writing down whatever came to mind, without trying to think too hard about it, correct mistakes, or even try to stay on task, not only did I enjoy those topics, I noted all sorts of interest ideas. It was a fantastic learning period. Writing about someone who had a positive influence on me was very engaging.

Yes, it does feel a little awkward when you first get started, staring at that blank screen not knowing where to begin. You may have to start with, "Today I did …" Or use a fun writing prompt. You may have to write a few thousand words of junk until you get comfortable with the process. However, with practice, ideas will literally leap to the fore. They will connect without you trying. You will have more to write about than you have time.

Yet another advantage is an "inventory" of the year. As we ended 2020, I'm sure people were taking stock of what they did and what they wanted to do.

"2020 was so shitty because of Covid! There was nothing good about that year!"

True, but I disagree as well.

I wouldn't be surprised if many people had to think long and hard about what they did throughout the year. What good things did they do? What goals did they check off the list? What did they accomplish despite the adversities? What should they look to do for 2021?

With a journal, those questions can be easy to answer. Keep a list of goals, Check those off. Write about the things you want to do. Write about the things you want to learn. Write down all the great things you did. Write down all the things you plan to change. Write down how you expect 2021 to play out. As we know, all the world's a stage. Make it your story.

There are a myriad of journal apps to choose from. The biggest thing people are looking for is a way to protect those thoughts and ideas. As mentioned, I use Scrivener with a password protected DMG file. No need to use a specific app or document format. Everything in the volume is protected. This has served me well for 5 years.

Whichever route you take, I find the journal invaluable. It's fun, educational, motivational, inspirational, and extremely satisfying. Regardless of platform, regardless of tool, take the time to write. Write down goals, thoughts, ideas, aspirations. In many cases, you'll see the plan develop and you can follow it to fruition.

Other articles of interest:

How and Why I Use 4 Monitors

I got my start with "multimon" back in the days of Windows 2000, perhaps a touch earlier. For certain I had two 19inch CRT monitors hooked up to my machine. I was so overwhelmed with how empowering this was. I could follow instructions on one monitor, then do them on the other. I could view web pages and watch a video. It was pretty amazing for it's time since it took two separate video cards to make the magic happen.

Years later, I upgraded to a 3 monitor setup. By this time video cards were powerful enough to handle multiple displays from a single card.

Now, with technology being what it is, I have 4 monitors connected to my Mac Pro using an RX 580 video card. Why? Primarily because I can. Four video outputs means four monitors. Using less would be wasteful. 🙂

Seriously, the 4 monitor setup is incredibly powerful and productive. Everything I need during the day is ready for me. I don't have to constantly switch between apps. I don't have to search for the window with the piece of information I need.

As I've mentioned in other articles, there are 4 32 inch monitors hooked up, 2 in a vertical orientation, 2 in the standard landscape orientation.

My main screen and the one to the right of it are landscape. On the second monitor is TaskPaper in a half screen size. I work in this for several hours a day. I have my ticket notes, the checklist of steps I've completed, the next step in the series, my test data, plus any other notes I need for my work.

Next, taking up a 1/3rd of the screen is CopyLess. This is a list of everything I'm pasting as test data. I don't have to keep opening apps or covering one with another, the data is right there ready for use.

Taking up the next 1/3rd of the screen is AYBO on top, with MiniNote underneath. I use AYBO to confirm calculations on the web site I'm testing. MiniNote is used to jot down quick bits I need for a few minutes. It's also the way I transfer notes, SQL code, JSON, and other information from one machine to another.

I have a completely separate iMac set up for testing. This is a fast way to push data over. I also have MiniNote on my home machine so links and notes show up there are well.

To the left of the main monitor are the two vertical monitors, side by side, slightly curving around. These are both used to display lots of daily information. Each screen has 3 quadrants.

At the bottom is Slack, so I can keep an eye on the conversations, including deployment notifications. On top of that is another browser so I can see two pages at the same time. I can be "admin" in one, then standard user in the other. At the top is a bit of whimsy at the moment with a seasonal video playing on a loop. It's a fireplace scene with the snow falling. It's visually appealing.

Monitor 4 is set up the same way with 3 quadrants. At the bottom is 2Do with reminders and tasks for the day. TextSoap sits above it, where I reformat text as needed, which happens a lot during the day. Above TextSoap is Messages so people can reach me through the Mac instead of email.

In many cases, the top quadrant of both monitors gets swapped out. Sometimes it's a chat client, sometimes a calendar, sometimes Mail, sometimes it's an image displayed in PhotoStickies.

There is a lot of information all at my fingertips. Everything I need for the day is ready. My main apps like Firefox, Mail, DevonThink Office Pro and DevonAgent are still full screen which is fine. I cycle through those as needed.

To keep track of the window positions, I have Magnet and Keyboard Maestro. Magnet handles the quick sizing such as "Left Half," as well as resizing to full screen. Keyboard Maestro handles layout and resizing of apps on the vertical monitors.

In this way I can resize LibreOffice Calc on the vertical screen for long spreadsheets. I can expand Skim for PDF reading. I can resize the browser to show a long table of figures on the page. I can bring windows to the fore, resize, full screen, then put them back using hotkeys. This keeps the desktop consistent.

What this gives me is an efficient way of getting my work done. I can see everything I need at a glance. My flow of productivity isn't interrupted by having to switch around apps or remember where I left something.

Most people will think this is overkill (it is) until they use it. Then they want to set up something similar for themselves.

A place for everything and everything in it's place.

Other articles of interest:

Recent Comments

  • How To Disable the Quicken Registration Prompt (32)
    • Mike: Hi, for anyone still trying to run older versions of Quicken, mine is 2004, I hope this helps. I managed to get it running on my old laptop by adding the three missing dll files, when I needed to upgrade I thought I would be able...
    • Juani: Hi, I have a registered Quicken 2016 and now my files are being held hostage by Intuit, cannot log in with my ID unless I upgrade therefore buy a subscription, I need to access my files, please HELP. Thanks
    • David: A BIG Thank you! I regularly reinstall Windows OS’s using different hardware configs. In short, Quicken 2011 (it’s March of 2020 as I write this) has been a program I’ve been very happy with and continue to use....
    • Karen L: Today I rang Reckon Australia again, got a different guy and he talked me through the process of getting me a new product key to enter since I already had the licence and it had been extended. Reckon have only extended it by 1...
    • Karen L: I have Australian Quicken 2008 on Windows 8.1. It keeps asking me to renew my licence which Reckon did for me however the activation won’t work. I tried this to disable the registration prompt however it did not stop the...
  • Enter dates into a date picker for Chrome and Firefox (1)
    • Prasanna: Hello, How to enter a previous month FIRST day and LAST day from the system date. Thanks Prasanna
  • Building Relationships with Developers (1)
    • Carlos Herrera: Ah yes. I have a team of software developers and it is genuinely so hard to keep them in the loop during projects. Though the more we work the better the communication becomes and we go from a trail and error process to a...
  • Integrating DevonAgent Pro with Alfred (1)
    • J. Garr: Beautiful, sweet, and simple. I love it; thanks for posting this solution.
  • Round and Round with the For..Next in Katalon Studio (1)
    • Sweet Ophaline Labador: Hello can you help me how to loop this scenario. Checking the elements is just the same. I want to check that the following function is available in Track Shipment – clicking on this can...
  • Parsing Strings in Katalon – Split, Substring and Readlines (6)
    • Don Pedro: String tempText=”Date Month January. Revenue $1,355,721.00″ println(tempText.split(“ \$”)[1]) You need to escape the $ with \$ The [1] on the split captures characters on the right of the split [0]...
    • Prasad: Hi, I need to get the substring from the below string ‘Date Month January. Revenue $1,355,721.00’ I want to extract text after $ value. Please help Thank you
  • Working with Dates and Date Formatting in Katalon Studio (15)
    • Sangeethaa: How to get previous day’s date(Yesterday’s date)? Actually I was using today.previous() before,it was working fine till today morning.Now its not retrieving right date. Can anyone please guide me on this.
    • Don Pedro: That could be a little tricky. First, is the text/format always going to be the same? You will need to do some parsing. Separate the first but of text at 2019. After that, split the text again to get rid of Central Standard...
  • Output status messages and test information by writing to the Log File Viewer in Katalon Studio (6)
    • Saranya: Good One. Could you plz share link to access all your katalon related blogs. ThankQ
    • Rodrigo Calabretta: I’m using the KeywordUtil.markFailed or KeywordUtil.markError and my test stops is being shown as Error and If I use in the @afterTestCase the testCaseContext.getTestCaseSta tus() to show the status test case...