Paul Sutton

Bash scripting 15 – Using data files 1

So, carrying on from the previous article. I am going to be using gnuplot and awk, and see what I can do with the data generated by my script.

So the first task, if of course to install gnu plot

sudo apt install gnuplot

It is also a good idea at this stage to determine version numbers of the tools we are using

gnuplot -V
gnuplot 6.0 patchlevel 0

and

awk -V
GNU Awk 5.2.1, API 3.2, PMA Avon 8-g1, (GNU MPFR 4.2.1, GNU MP 6.3.0)
Copyright (C) 1989, 1991-2022 Free Software Foundation.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.

Tags

#Bash,#Bashscripting,#Files,#UsingDataFiles,#Data

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Windows Recall

• Microsoft are bringing back their Recall feature

There are serious security and privacy concerns about this feature, and even if you opt out of it, it does not mean your data is safe.

Time to switch to replacement operating systems and start to learn them. Don't forget also that Windows 10 support ends in August. You may be forced to upgrade to Windows 11 on your hardware, or depending on when you bought the computer, upgrade the whole computer with a new one just so you can run Windows 11.

Links

• Linux Mint
  • Linux Mint Matrix
• Devon & Cornwall Linux user group
  • Devon and Cornwall matrix

Tags

#Microsoft,#Windows,#Recall,#Privacy,#Security

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Bash scripting 14 – Generating data files

Issue 249, May 2025 of Linux Magazine, p52-55, had an article on using awk, and gnu plot to manipulate data. In order to try and make use of this, and learn more, I am trying to figure out how to write a csv file.

Write 10 lines to the first column should be in sequence e.g. 1-10 and the 2nd column should be random numbers. This should be written to a file.

So after some digging I managed to come up with this, by modifying some existing solutions

write_csv(){
    echo \"$1\",\"$2\" > log.csv # write to file, use >> to append
}

#sources of help
echo "number" "data" # for reference, not written to file 
for ((i = 0 ; i < 10 ; i++)); do
    echo $i, $RANDOM # echo to screen (stdout)
	echo $i, $RANDOM >> log.csv # write to file
    #write_csv $(($i, $RANDOM % 10))
done

The screen output for me was

number data
0, 17109
1, 2872
2, 22572
3, 7228
4, 27923
5, 6335
6, 20004
7, 32343
8, 10005
9, 21905

While the file output (log.csv) was similar but without the headings, which I just put in for my own reference, the file does work as a csv as I can open in LibreOffice Calc.

0, 17109
1, 2872
2, 22572
3, 7228
4, 27923
5, 6335
6, 20004
7, 32343
8, 10005
9, 21905

As the numbers are quite big, a closer examination, promoted me to make the following modification

echo $i, $((RANDOM % 20)) # echo to screen (stdout)
echo $i, $((RANDOM % 20)) >> log.csv # write to file

This fixes the 2nd column to produce a number between 1 and 20.

number data
0, 17
1, 12
2, 12
3, 16
4, 2
5, 13
6, 10
7, 6
8, 16
9, 16
cat log.csv
0, 3
1, 17
2, 8
3, 16
4, 10
5, 18
6, 5
7, 14
8, 11
9, 11

The script was called gencsv1.sh, which as expected needed execute permissions

chmod +x gencsv.sh

References

In order to help me I used the following, so giving credit to these sources as would be expected.

• https://stackoverflow.com/questions/40175868/how-to-create-csv-file-using-shell-script
• https://stackoverflow.com/questions/1194882/how-to-generate-random-number-in-bash
• https://linuxhandbook.com/bash-loops/

Tags

#Bash,#Bashscripting,#Files,#Random,#Data,#CSV,

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Top 10 skills in demand 2025

There are numerous sites with this or similar information on what are the most in demand skills of 2025.

Some of my recent posts point to courses or information on these areas. Others will require your own research [2]. This list is from True Digital Academy [1].

1. Data Science & Cloud Computing
2. Artificial Intelligence (AI) & Machine learning (ML)
3. Big Data
4. Digital Marketing and strategy
5. Process Automation
6. Business Development
7. Digital Transformation
8. Information Security and Cybersecurity
9. Software and Application development
10. Internet of things

I made a post a few years ago, with a link to training courses from the Linux Foundation and others [3]. So there are some links that are related to the above list.

References

1. true digital academy
2. DuckDuckGo
3. Linux Foundation Training post
4. Linux Handbook

Tags

#Skills,#Year2025,#Demand,#Work,#Employment.#Training

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

YouTube removes 'gender identity' from hate speech policy

Why are you using platforms such as Youtube? By using Youtube you are using a platform that clearly does not support equality, diversity and inclusion in order to please the US government.

SCOOP: YouTube removes 'gender identity' from hate speech policy. “YouTube quietly removing ‘gender identity and expression’ from its list of protected groups is a major radical shift away from best practices in the field of trust and safety.”

• YouTube removes 'gender identity' from hate speech policy

Time to switch to peertube

Tags

#Youtube,#Policy,#DEI,#Peertube

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

AI Footprint

This is well-known about, but not enough discussion around the subject.

Links

• The attribution problem of a seemingly intangible industry
• Institute for Science and Ethics (IWE)
  • IWE Mastodon
• 16/3/2025 – Related Trump Plan to Use EPA to Promote AI Could Harm the Climate and Drive Up Electric Bills
  
• Elon Musk Reportedly Doing Something Horrid to Power His AI Data Center

Tags

#ArtificialIntelligence #AI #Sustainability #Environment #ClimateChange #SustainableAI #GreenTech #TechForGood #Innovation #ResponsibleAI #DigitalTransformation #EthicalAI #RenewableEnergy

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Bash scripting 13 – Linux permission table

As previously mentioned, scripts need to be given permission to run. This table should help with the process of working out what permission to set on a file or script.

I think the usual default is 755 which gives the user read, write, execute. Both the group and other users of the system get read and write.

For the most part, you can set execute permission on a script with

chmod +x scriptname.sh

	READ			WRITE			EXECUTE
USER	GROUP	OTHER	USER	GROUP	OTHER	USER	GROUP	OTHER
400	40	4	200	20	2	100	10	1

The markdown code for the above is

|         | **READ** |         |         | **WRITE** |         |         | **EXECUTE** |         |
|:-------:|:--------:|:-------:|:-------:|:---------:|:-------:|:-------:|:-----------:|:-------:|
|  _USER_ |  _GROUP_ | _OTHER_ |  _USER_ |  _GROUP_  | _OTHER_ |  _USER_ |   _GROUP_   | _OTHER_ |
| **400** |  **40**  |  **4**  | **200** |   **20**  |  **2**  | **100** |    **10**   |  **1**  |

Which may be useful. I used Tables Generator to help create this.

Chat

I am on the Devon and Cornwall Linux user group mailing list and also their matrix channel as zleap, it is better to ask there, that way others can answer too.

Tags

#Bash,#Bashscripting,#Files,#Permissions.#chmod,#Read,#Write, #Execute

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Devon needs AI skills for growth

BBC NEWS:

Interesting article from BBC news on 9th April, Devon needs AI skills for growth. The Devon & Somerset Local Skills Improvement Plan are looking at ways to help people develop the skills for the future.

I think AI can sort of be put in to 3 boxes

• Machine Learning (ML)
• Generative AI (GenAI)
• Large Language Models (LLM)

I have put a Blog post on Linux foundation training which has some links, to courses run by the Linux Foundation, EdX and others and in fact EdX, offer a number of AI courses, but as everything needs something (usually a server running Linux) to actually run on, it seems that learning how to use, maintain and administer GNU/Linux operating systems is also essential. There are a lot of AI software / Tools or Libraries under a GNU or similar licences anyway.

It could be an opportunity for the Devon and Cornwall GNU/Linux user group to expand and gain new members, who can at least get help and advice to get started, even though it is much easier to install than it was 25 years ago.

Tags

#Devon,#SomerSet,#Skills,#AI,#LLM,#GenAI,#ML,#Training,#Jobs,#Linux,#GNU,#GNULinux.

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Bash scripting 12 – Files and Grep

Rather than make a video for this, I decided to just make a blog post so that I could include downloadable or at least copy / pasteable components.

Grep stands for GNU Regular Expression Parser, In essence and among other things, it can read (or parse) a file and report on contents, or in the case of this, find a specific string of text.

lorem Ipsum, is standard in the printing industry as it is dummy text used to fill on a page. I have pasted below an example and it just happens to explain further.

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.~

If you copy and paste the above and save in a text file called lorem.txt we can do some neat stuff with grep and a few other commands. I am not an expert at this, so this is some of what I picked up while researching this post.

Firstly we can find a specific word (or string) in the text with

cat lorem.txt | grep the

we can also do

grep the lorem.txt

Both will search the file lorem.txt, display the file contents and highlight the word 'the' from the text.

This is great, so what else can we do

In a short file, the number of times a word may appear may be less than 5 or 10. So we could just count manually. As discussed in a previous video, the command wc or word count, does what it says and counts the number of words.

So I found the following

cat lorem.txt | grep -o the | wc -l

Which gives the output as 6 which is how many times the word 'the' appears in the text.

As with other commands, there is a man page so

man grep

and

man wc

Should provide useful information, you can also search for information with duckduckgo and there are numberous tutorials on line.

Hope this is useful

Chat

I am on the Devon and Cornwall Linux user group mailing list and also their matrix channel as zleap, it is better to ask there, that way others can answer too.

Tags

#Bash,#Bashscripting,#Files,#TextSearch,#StringSearch,#Grep,#wc,#WordCount

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.

Open Street map 4

So, further to my previous post, A little disappointed, but not surprised in the lack of interest in people wanting to help with this.

Suffice to say, I have made my first edit, which is to add the Steam café & Discovery Centre to the Map.

I have also found a nice guide on how to edit.

I will endeavour to add more. Furthermore, I also added the music shop next door as this is no longer Lucky Dip, as it has changed hands, but no new shop name has been added to the front of the shop. This is fine, OSM is about local knowledge.

I have also received some post card size flyers from OSM, so will leave some in the STEAM café, and elsewhere too.

You should be able to tag me on OSM/Forum and Matrix with @zleap

Links

• Open Street Map
• Street Complete
  • Street Complete App (Android)
  • JOSM Software for map editing
  • Open Street Map UK Community
• OSM Profile
• Matrix Room
• Mastodon

Tags

#Community,#Open,#StreetMap,#Cartograpy,#CrowdScource, #OpenStreetMap

---

Mastodon	ShellLabs	Join Mastodon

AI statement : Consent is NOT granted to use the content of this blog for the purposes of AI training or similar activity. Consent CANNOT be assumed, it has to be granted.