Digital Process Automation

Many people work a significant amount of their time in front of a computer today. And for many there are repetitive tasks. If the task is shared by many people, this is a chance to develop dedicated software for it. Tabular calculations (excel), writing documents (word), communicating (e-mail / chat software) are some examples.

But there are other tasks which are not so common. Which are too specialized so that it's not worth to pay a developer for designing, programming, testing, maintenance; to pay for marketing, support, legal fees, and infrastructure.

This is the part where it becomes interesting to learn to use some tools.

Text Editors

My favorite editor is Sublime Text hence the links / names in the following are for it. But there are a couple of other editors like Atom or Visual Studio Code that might have the same features.

Suppose you have a long list of names, phone numbers and e-mail addresses which was provided by your "contacts" export:

Martin Thoma; +49 123 456; [email protected]
Adam Aaron;+1 23 456 789; [email protected]
Berta Booth;002 345678; [email protected]
...

You want to insert the e-mail addresses in another system. But for that you need to have only the e-mail addresses in double quotes ". How do you get that?

You could, of course, just manually copy all of the e-mail addresses to another file. Assuming you have maybe 80 of those and it takes you 5 seconds for each entry in average, you would need less than 7 minutes. That is ok. But it is boring. And this kind of task will come more than once in your live.

So, instead, you can learn a tiny bit about regular expressions. They are a way to define patterns in text. For example, . matches any single character except a newline. With + you say that you want at least one of the things you defined before the +. If you have the combination .+, it means that you want to match any single character and as many of them as possible. By adding a ? you say you want to match as few as possible. Hence the search .+?;.+?; means that you want to find any characters, then a ;, then again any caracters until the next ; comes. In my editor it looks like this:

3 matches for the RegEx .+?;.+?;
3 matches for the RegEx .+?;.+?;

Then, I press Ctrl+Shift+L to get multiple cursors. From this point on everything I enter will be applied to all matches. First, you can delete the name and phone number. Then you enter " to make the opening double quote, then you press End to get to each end of the line. You press " again to get the closing double quote. Task solved.

For 80 entries, this would have reduced the time from 7 minutes of an annoying repetitive task to about 15 seconds.

For more about regex, you might want to look at a cheat sheet or go through a tutorial. Another cool thing is to enter different things with multiple cursors, e.g. a sequence of numbers. Have a look at Text Pastry for that.

Shortcuts

All operating systems and most programs define some keyboard shortcuts. If the programs are really nice (like Sublime Text), you can re-define the shortcuts yourself. This can save so much time. Some shortcuts I use all the time:

Operating System

Switch Windows
Alt+Tab
Switch Workspaces
Ctrl+Alt+[arrow]
Close Window
Alt+F4
Lock Screen
Ctrl+Alt+L

Programs

Save
Ctrl+S
Open
Ctrl+O
Switch Tabs
Alt+[number]
Execute (Sublime)
F5

Shell and Scripting

A little bit of knowledge of a shell is helpful. For Linux, the most common shell is called bash. Knowing some command line tools and how to execute them in simple shell scripts is pretty powerful.

Here are some of the basic commands, roughly in the order how often I need them:

Change Directory
cd [relative or absolute path]
Find within a file
grep
Look up what you execute in the past
history
List files/directories
ls
Open a file
xdg-open
Move a file
mv origin destination
Copy a file
cp origin destination
Delete a file
rm filepath
Disk usage
df -h and ncdu (website)
Running processes
top and htop
Repetedly execute something
watch
Show content of a text file
cat, head, tail
Word / line / character counts
wc
Locating files
locate, find
Read the manual
man
Print working Directory
pwd

And some things you have to install

Show differences between text files
meld file1 file2 (website)
Download a webpage
curl or wget or httpie (website)
Manipulate JSON files
jq (website)
Environment Variable Loading / Unloading
direnv

vim and its various plugins like NERDTree and Powerline are a completely seperate topic.

See also:

Screen automation

Some times tools don't come with the necessary interfaces to automate things. The simplest example are IDLE games like Cookie Clicker. In this case you just have to click. Fast. All the time.

So what you can use is xdotool.

First, navigate to where you want to click. Leave your cursor there and execute xdotool getmouselocation. Once you know the location, you can move there with xdotool mousemove 100 200 and execute a left click with xdotool click 1.

So a script for this could be:

#!/bin/bash
for i in {1..500}; do
  xdotool click 1 &
  sleep 0.001
  echo $i
done

Published


by

Category

Cyberculture

Tags

Contact