Paul Sutton

Grep

Bash scripting 12 – Files and Grep

Rather than make a video for this, I decided to just make a blog post so that I could include downloadable or at least copy / pasteable components.

Grep stands for GNU Regular Expression Parser, In essence and among other things, it can read (or parse) a file and report on contents, or in the case of this, find a specific string of text.

lorem Ipsum, is standard in the printing industry as it is dummy text used to fill on a page. I have pasted below an example and it just happens to explain further.

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.~

If you copy and paste the above and save in a text file called lorem.txt we can do some neat stuff with grep and a few other commands. I am not an expert at this, so this is some of what I picked up while researching this post.

Firstly we can find a specific word (or string) in the text with

cat lorem.txt | grep the

we can also do

grep the lorem.txt

Both will search the file lorem.txt, display the file contents and highlight the word 'the' from the text.

This is great, so what else can we do

In a short file, the number of times a word may appear may be less than 5 or 10. So we could just count manually. As discussed in a previous video, the command wc or word count, does what it says and counts the number of words.

So I found the following

cat lorem.txt | grep -o the | wc -l

Which gives the output as 6 which is how many times the word 'the' appears in the text.

As with other commands, there is a man page so

man grep

and

man wc

Should provide useful information, you can also search for information with duckduckgo and there are numberous tutorials on line.

Hope this is useful

Chat

I am on the Devon and Cornwall Linux user group mailing list and also their matrix channel as zleap, it is better to ask there, that way others can answer too.

Tags

#Bash,#Bashscripting,#Files,#TextSearch,#StringSearch,#Grep,#wc,#WordCount

Mastodon ShellLabs Join Mastodon
AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Donate using Liberapay

CyberSecurity part 13

So we are on part 13, well done and thank you to everyone who has stuck with this so far.

So today, I am going to link to three videos, which give more details on specific bash commands. Hopefully this will speed things up a little, there are probably better videos, so please feel free to search, watch and share alternative videos.

Links

Tags

#CyberSecurity,#Shell,#Grep,#Sed,#Cron

Mastodon ShellLabs Join Mastodon
AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Donate using Liberapay

Ansi Weather

Donate using Liberapay

Getting weather information is really useful. What happens if you're at the command line in Linux? I found a really little application that can help

ansiweather

apt install ansiweather

ansiweather -l Plymouth, UK

Ansi Weather Output

So what else can we do with this

  1. Send the output to Mastodon with toot post

This is a two step process

  1. ansiweather -l Plymouth, UK > weather.txt
  2. toot post < weather.txt

Will send the weather info to Mastodon.

However this does not include any date info

We can fix this with

  1. date > weatherinfo.txt
  2. ansiweather -l Plymouth, UK >> weatherinfo.txt

then send the whole lot to Mastodon with

  1. toot post < weatherinfo.txt

So, if we put this in to a final shell script we need:-

#send weather info to Mastodon
# current date
date > weatherinfo.txt
# current weather
ansiweather -l Plymouth, UK >> weatherinfo.txt
#send to Mastodon
toot post < weatherinfo.txt
# done
echo done

Again released under GPLv3

I tried to get festival to speak the weather, it is not perfect but this sort of works, you will need to direct to weather.txt first.

festival —tts < weather.txt

Looking in to this further, the issue is the brackets etc, so this stackoverflow post

strips out the colour formatting

sed 's/\x1b[[^\x1b]*m//g' weatherinfo.txt

Therefore

sed 's/\x1b[[^\x1b]*m//g' weatherinfo.txt > weatherinfo2.txt

Sends the newly formatted text to weatherinfo2.txt

So running back through festival

festival —tts < weather.txt

Is perhaps a little better, but not perfect

So going back to what we wrote earlier to send to Mastodon, the new script

  1 #send weather info to Mastodon
  2 # current date
  3 date > weatherinfo.txt
  4 # current weather
  5 ansiweather -l Plymouth, UK >> weatherinfo.txt
  6 # clean up output with sed
  7 sed 's/\x1b\[[^\x1b]*m//g' weatherinfo.txt > weatherinfo2.txt
  8 #send to Mastodon
  9 toot post < weatherinfo2.txt
 10 # done
 11 echo done

Produces much nicer output. The top bottom part of this illustrates what was sent before we stripped out the colour formatting

However it still isn't perfect, as it removes part some of the wording, but it is hopefully getting there.

Weather Output (new)

REFERENCES

TAGS

#YearOfTheFediverse,#Weather,#Scripting,#Bash,#Linux, #Mastodon,#ProblemSolving,#AnsiWeather,#programming, #Stackoverflow,#sed,#cat,#grep,GPL3,#FSF

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

Mastodon ShellLabs Join Mastodon
AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Donate using Liberapay