draft0 - a shared blog by just some people

Go To Navigation
Show/Hide Navigation

Entries tagged 'cat:Code' (Page 1)

backup.sh
This entry is an update of the entry 'Backup Shell Script'.

I've updated/improved my backup script again.

Download


#!/bin/bash
#2021-11-03

budir="/mnt/backup"
num=8
list="/root/backup-list"
hostname=$(</etc/hostname)
srcdir="$1"
#NAME=${SRCDIR//\//--}
name="$2"

function usage () {
  echo "This script requires either one or two arguments."
  echo "Usage:"
  echo "$0 JOBLIST"
  echo "$0 SOURCE_DIRECTORY NAME"
  echo
  echo "JOBLIST File that contains one backup job per line in the format SOURCE_DIRECTORY NAME"
  echo "SOURCE_DIRECTORY Directory that should be backed up. Please, no trailing slash."
  echo "NAME A string that is used to name the backup in the destination."
  exit 1
}

function single() {
  if grep -qs "$budir " /proc/mounts
  then
    fullname="$budir/$hostname/$name"
    printf "\n"
    printf "Attempting backup of source directory '%s' to '%s'.\n" "$srcdir" "$fullname"
    printf "Number of differential backups to keep: %s\n" "$num"
    printf "Removing oldest backup... "
#    rm -rf "$fullname.$num"
    rsync --archive --delete "$budir/empty/" "$fullname.$num/"		# This is quicker than rm.
    printf "Done.\n"
    for ((i=$num; i>=2; i--)); do
      printf "Renaming '%s' to '%s' ... " "$fullname.$((i-1))" "$fullname.$i"
      mv "$fullname.$((i-1))" "$fullname.$i"
      printf "Done.\n"
    done
    printf "Duplicating last backup ('%s' to '%s')... " "$fullname.0" "$fullname.1"
    [[ -d $fullname.1 ]] && exit 1							# This directory should not exist at that point.
    cp -al "$fullname.0" "$fullname.1"
#    rsync --archive --acls --xattrs --hard-links "$fullname.0/" "$fullname.1/"
    printf "Done.\n"
    now=$(date)
    printf "\n\n\n" >> "$fullname.log"
    printf "STARTING INCREMENTAL BACKUP AT %s\n" "$now" >> "$fullname.log"
    printf "Starting new incremental backy uppy at '%s.0'..." "$fullname"
    if rsync --exclude ".cache" --archive --no-links --delete "$srcdir/" "$fullname.0/" 2>&1 | tee -a "$fullname.log"
    then
      printf " Done.\n"
      printf "Done.\n" >> "$fullname.log"
    else
      printf " Failed.\n"
      printf "Failed.\n" >> "$fullname.log"
    fi
    now=$(date)
    printf "\nIf this line is here the script finished (with or without errors) at %s\n" "$now" >> "$fullname.log"
  else
    printf "'%s' is not mounted. Aborting." "$budir"
    exit 1
  fi
}

function fromlist() {
  while read job; do
    if [ -n "$job" ]; then
      name="${job#* }"
      srcdir="${job% *}"
      single
    fi
  done < "$list"
}

mkdir "$budir/empty"									# Empty directory for a quicker method to delete a large directory.

case $# in
0)
  mount "$budir"
  printf "Reading backup jobs from list. Defaulting to %s.\n" "$list"
  fromlist
  ;;
1)
  mount "$budir"
  list="$1"
  printf "Reading backup jobs from %s." "$list"
  fromlist
  ;;
2)
  mount "$budir"
  single
  ;;
3)
#  I can't decide whether to make the third argument num or budir. I don't need it anyway.
#  num=$3
#  budir=$3
#  mount $budir
#  single
  usage
  ;;
*)
  usage
  ;;
esac

exit 0
Comment via email
SBWG - The Pathshortener And Other Recent Changes

I made it my goal to harden SBWG before I start to implement new features. Before I call the next version of this project 1.0 I want to make sure that unexpected input from the command line or from source files, absurd numbers of absurdly long tags and content items, stupidly weird filenames or random binary data as tag values as well as purposfully created traps in the various places where input is processed are handled well, meaning that nothing fails unless there is no sensible way around it, and if something fails, that nothing breaks. Data should be filtered carefully, errors should be handled well and whereever possible data should be made processible if it was supplied in an unprocessible form to reduce the chances of errors. On top of that I wanted to make sure that the script did its job in a reasonable amount of time considering the circumstances. I mean, it will never be very fast. Bash is just not the right language for that. But there certainly were some repetitive tasks that could be improved. Fot the latter I created a simple caching functionality that will probably be extended in the future. I managed to reduce the (calculated/estimated) generation time of my biggest test web site from almost 300 years to a few days. Actual web sites will of course not take that long to generate, even on a slow machine. A huge web site will maybe take up to one day to generate completely, even without the new options that keep the script from re-generating existing unchanged parts of the web site. But before sombody will try to create such a big web site with SBWG I will probably have improved speed further. And even then it's a worst-case time.

As part of the aforementioned goals I have started working on last new feature before version 1.0. I call it the pathreducer. Since many of the files created by SBWG are named after the tags they represent, they can become quite long and contain almost any printible character, including multi-byte unicode characters or characters of character sets I haven't even heard of. I definitely don't want to restrict more than I already have what characters and how many of them tag values can contain. Especially filesystems used by operating systems from Microsoft are relatively restrictive in maximum allowed directory, path and filename length and allowed characters. By default the pathreducer is not used. But if enabled via command line option or in a web site's settings file, it will filter directory and filenames and shorten them to a user-defined maximum length. If the pathreducer decides to change a path elements it also adds a 6-character hash value to make shortened or otherwise reduced path elements as good as unique.

That works well for now and even can create 8.3 or 6.3 filenames for old DOS filesystems. But the result is not very nice because it isn't aware of what filesystem it is going to write a file to. To be save it removes more characters than it would have to for ext and NTFS filesystems. In the future I may extend the pathreducer to detect the filesystem at least of the root of the output directory automatically and decide how exactly path elements should be reduced according to the actual limitations of the present filesystem. Than it may even be enabled by default, even though it can increase the generation time quite a bit.

There are still some tests that I want to do and I will probably find some more things that I want to fix before version 1.0. But I see light.

Comment via email
Generating Bitmap Files With Bash

I needed a large amount of image files to try something. I wanted them to be different images. But what's in them didn't matter. So I looked at a BMP file to see how I could create one byte for byte automatically. Bitmap is just the first uncompressed format that I thought of. This is what I came up with:


rbmp() {
  echo -n -e '\x42\x4D\x2A\x02\x00\x00\x00\x00\x00\x00\x7A\x00\x00\x00\x6C\x00\x00\x00\x10\x00\x00\x00\x09\x00\x00\x00\x01\x00\x18\x00\x00\x00\x00\x00\xB0\x01\x00\x00\x23\x2E\x00\x00\x23\x2E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x42\x47\x52\x73\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x'
  c=$(</dev/urandom tr -dc '0123456789ABCDEF' | head -c864 | sed 's/.\{2\}/&\\x/g')
  echo -n -e "$c"'00'
}

Or, if you would like to do run a command for each pixel/color before it is generated, you can do it like this:


# Create a 24 bit bitmap file of 16x9 randomly colored pixels
randombmp() {
  echo -n -e '\x42\x4D\x2A\x02\x00\x00\x00\x00\x00\x00\x7A\x00\x00\x00\x6C\x00\x00\x00\x10\x00\x00\x00\x09\x00\x00\x00\x01\x00\x18\x00\x00\x00\x00\x00\xB0\x01\x00\x00\x23\x2E\x00\x00\x23\x2E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x42\x47\x52\x73\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
  i=1
  while (( i < 433 )); do
    c=$(</dev/urandom tr -dc '0123456789ABCDEF' | head -c2)
    echo -n -e '\x'"$c"
    i=$((i+1))
  done
}

I'm sure I did something wrong. I just copied the header of some BMP file with the right format that I don't even know what it was created with. I didn't actually look up how the header of a BMP file is composed. But it worked.

Comment via email
USB/Serial PWM Fan Controller Using an Arduino

I wanted to be able to control the speed of the fans in my big NAS, Fred, individually. Even though the mainboard in use has five PWM fan connectors, the chipset can only control the speed of all fans together. There are probably good fan controllers commercially available that solve this problem better than I did. But they seemed overpriced and it seemed like a fun learning project for me.

The fan controller that I made uses an Arduino Nano clone that listens to it's serial port, waiting for a command to change the speed of a fan. When a command is recognised the continuous PWM signal for that fan is changed accordingly. It's possible to control up to six fans this way with an Arduino Nano. I'm only using three though since I only have three fan groups that need to be controlled separately.

The Arduino sketch/C code for the Arduino Nano that I used is as follows.




//fan speed sensor wire attached to digital pin 2 with a 10kohm pullup resistor
//fan PWM control wire attached directly to digital pin 9

#include <PWM.h> //include PWM library http://forum.arduino.cc/index.php?topic=117425.0

volatile int half_revolutions1; //allow half_revolutioins to be accesed in intterupt
volatile int half_revolutions2; //allow half_revolutioins to be accesed in intterupt
int rpm1; //set rpm as an integer
int rpm2; //set rpm as an integer
int pwm=255;
const byte numChars = 5;
char receivedChars[numChars];

boolean newData = false;

void setup()
{
  InitTimersSafe(); //not sure what this is for, but I think i need it for PWM control?
  bool success = SetPinFrequencySafe(9, 25000); //set frequency to 25kHz
  pwmWrite(9, 51); // 51=20% duty cycle, 255=100% duty cycle

  pinMode(5, OUTPUT);
  pinMode(6, OUTPUT);
  analogWrite(5, 170);
  analogWrite(6, 255);
  pinMode(2,INPUT_PULLUP); //set RPM pin to digital input
  pinMode(3,INPUT_PULLUP); //set RPM pin to digital input
  half_revolutions1 = 0;
  rpm1 = 0;
  half_revolutions2 = 0;
  rpm2 = 0;

  Serial.begin(9600);
}



void loop()
{
  sei(); //enable intterupts
  attachInterrupt(0, fan_rpm1, RISING); //record pulses as they rise
  delay(1000);
  detachInterrupt(0);
  attachInterrupt(1, fan_rpm2, RISING); //record pulses as they rise
  delay(1000);
  detachInterrupt(1);
  cli(); //disable intterupts

  rpm1 = (half_revolutions1/2)*60;

  Serial.print("1");
  Serial.println(rpm1);

  rpm2 = (half_revolutions2/2)*60;

  Serial.print("2");
  Serial.println(rpm2);

  rpm1 = 0;
  half_revolutions1 = 0;

  rpm2 = 0;
  half_revolutions2 = 0;

  pwm = 255;
  recvWithStartEndMarkers();
  processCommand();
}

void fan_rpm1()
{
  ++half_revolutions1; //increment before returning value
}


void fan_rpm2()
{
  ++half_revolutions2; //increment before returning value
}


void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = 's';
    char endMarker = '\n';
    char rc;
 
    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void processCommand() {
    if (newData == true) {
        Serial.print("s");
        Serial.println(receivedChars);
        switch (receivedChars[0])
        {
            case '1':
                receivedChars[0] = '0';
                sscanf(receivedChars, "%d", &pwm);
                analogWrite(5, pwm);
                break;
            case '2':
                receivedChars[0] = '0';
                sscanf(receivedChars, "%d", &pwm);
                analogWrite(6, pwm);
                break;
            case '3':
                receivedChars[0] = '0';
                sscanf(receivedChars, "%d", &pwm);
                
                break;
//            default:
//                Serial.println("I don't know what that means.");
        }
        newData = false;
    }
}

Well, how should I put it? It works, usually.

(tbc?)

(tba:photos)

Comment via email
SBWG 0.9.6

As foreseen I've made slow progress in development of SBWG, the script that generates this web site, because I want to test and improve it with the current feature set before I start to implement new features. The former is necessary. The latter is more fun. But today I've reached a point where I can say that the only thing left to do before I call it v1.0.0 is testing edge cases and things that I didn't think of testing before, as well as fixing potential bugs discovered from this testing.

So, version 0.9.6 is out, everything is working, The README file, other documentation included in the package, the example website, the included style sets and partly even the code quality has been deemed satisfactory, and I hope I'll find the time to test all sorts of weird stuff and discover and fix some bugs next month, at which point version 1.0.0 will be published and I'll finally be able to allow myself to start working on new features, some of which are awaited by both regular users of the script.

Comment via email
Backmatch - A Simple Dual-N-Back-Inspired Performance Task Trainer For Bash

Here is a script that I wrote as a short side project because I wanted my own n-back trainer. I'm aware that the practical memory improments of n-back training, even when using a proper audiovisual dual n-back trainer, is not as great as it's often said to be. I just wanted to try it and see whether I like the training.

Usage

Execute the script and pass it a number that resembles the difficulty level. For example bash backmatch 3 starts the script with a 3-back task. On your keyboard press the key of the letter that was displayed n letters ago (in this example 3 letters ago). When you press a key the next letter is presented immedietly. If you don't press a key for 3 seconds (the time value can be changed by changing the variable sec) the next letter is presented and you don't get a point for this letter. When you exit the script by pressing ctrl+c your keypresses get compared to what was presented and you get your score.


#!/bin/bash

if [[ ${#} -ge 1 ]] && [[ ${1} =~ ^[0-9]*$ ]]
then
  n=${1}
else
  echo "'${@}' is not a number, is it?"
  exit 1
fi

echo "Ctrl+C pressing is for quitters."
npo=$((n+1))
sec=3

end() {
  echo -en "           \n"
  if [[ ${#str} -ge ${npo} ]]							# If enough characters had been generated
  then
    for i in $(seq $n); do echo -n "-"; done					# offset by $n dashes
    echo ${str}
    echo ${you}
    got=0
    for i in $(seq ${#str})
    do
      [[ ${str:$i:1} == ${you:$((i+n)):1} ]] && got=$((got+1))
    done
    echo -e "\n${got} out of $((${#str}-npo)) correct"
  else
    echo "Not enough data to judge you."
  fi
}

trap end EXIT

while true
do
  str+=$(cat /dev/urandom | tr -dc 'A-Z' | head -c 1)				# Get a random new letter.
  echo -en "\r       \r${str: -1}"						# Print the last character in the string (the new letter).
    read -n 1 -t ${sec} key							# Get a single character input, timeout $sec seconds.
    [[ -n ${key} ]] && you+=${key^^} || you+="-"
    [[ ${key^^} == ${str: -$npo:1} ]] \
      && echo -en "\r       good" \
      || echo -en "\r        bad"						# Check if uppercase input char is the same as the nth char from the back.
done

end

exit 1
Comment via email
Setting Screen Brightness To Any Value With A Three-Step Keyboard Shortcut (Linux)

Initially out of necessity because the brightness keys of my new laptop didn't work out of the box (the driver was aded to my distro not a month later, which should have been acceptable, but I didn't know that at first), I was looking for a way to set the backlight brighness of my laptop's internal screen easily, without typing a command in a shell. What I ended up using I like even better than the usual + and - keys.

I'm using i3wm and dmenu. The way I set screen brightness is

  • 1) I enter the shortcut (mod+B in my case)
  • 2) I enter a number and
  • 3) I hit Enter.
  • It's simple to implement. Just put this line in the i3 config file:

    
    bindsym $mod+b exec \
      thatbright=$(echo "1000\n2000\n3000\n4000\n5000\n6000\n7000\n100\n10" \
      | dmenu -p "How bright though?") && echo $thatbright \
      > /sys/class/backlight/*/brightness
    

    You can put it into one line (without the \s inbetween) if you want.

    You could easily change that to a two-step or single shortcut if you like. I like the three-step version because it allows me to choose from one of seven brightness modes easily but also lets me enter a value below or between those pre-sets without taking up more than one key.

    Code Explanation

    First, the variable thatbright is set to the number that dmenu outputs, which can be one of the numbers that are echoed to the pipe (selected with arrow keys or completed when typed in dmenu) or another number that is entered into dmenu. If that was suggessful, the value is written to /sys/class/backlight/*/brightness. If you have multiple backlights in /sys/class and you only want to set one or if your shell doesn't support wild cards in paths, you can change the * to whatever applies to your system, e.g. intel_backlight.

    Scripts/Commands to set the screen brightness

    The simplest script that sets the screen brightness in Linux is probably the one-liner from above: echo $1 > /sys/class/backlight/*/brightness. This sets the raw numerical value. You need to know what a sensible range of numbers is and what the maximal accepted value is (look at '/sys/class/backlight/[YOUR_BACKLIGHT]/max_brightness'). But there are more elaborate scripts, like bbacklight by Giuseppe Eletto with which you set the brightness with a percentage value.

    Comment via email
    SBWG 0.8.10

    So, I'm still making slow process with SBWG. I had more fun with it when I was out and implementing new features. But I find it important to finish version 1.0.0 with the currently defined set of festures and goals, which include finishing documentation, testing and code hardening, which are less fun for me.

    I've always treated the third level of the version number (x.x.thisone) as a means to declare a new version done when I feel like having achived something. So today I declare version 0.8.10 as done. There really isn't much left to do to meet my milestones for v0.9.0. And from there on it will only be testing and possibly a little bit of code improvements to get to my set goals for v1.0.0.

    I'm looking forward to this not only because I'll like the feeling of having achived a goal, but also it will mean that I'll be free again to introduce new features. I still have more ideas than necessary about what to do with SBWG.

    But right now I'm enjoing the fact that I'm able to make myself believe that it's okay to move on as slowly as I want and let my colloquial executive dysfunction do its think without impacting my feeling of self-worth oo much.

    Comment via email
    Mastodon