Entries tagged 'cat:Scripts' (Page 3)

Automatic backups with rsync under Linux

Entry Permalink (edited 2020-09-27) Authors: steeph (337) Categories: Backups (4) Bash (30) Code (30) Computer (75) Linux (34) Projects (40) Scripts (27) Software (50) obsolete (4) Languages used: en (189) Topics: Projects → Code (26)

There is a newer version of this entry: 'Backup Shell Script'.

This is a very primitive and bad script. I've posted a new version here.

"No backup no pity!"

I don't agree. But I like the saying for reasons that I shall not analyse here right now.

I don't always have proper backups of my data, let alone off-site backups. I have no backup at all of a large portion of my hoarded data. But I did at some point take the time to create scripts that back up important stuff automatically. Here is a simple one using rsync and a mounted share as a target.


#!/bin/bash

NAME="usershomedir"
SRCDIR="/home/user"
BUDIR="/mnt/backup/"

rm -rf $BUDIR$NAME.8
mv $BUDIR$NAME.7 $BUDIR$NAME.8
mv $BUDIR$NAME.6 $BUDIR$NAME.7
mv $BUDIR$NAME.5 $BUDIR$NAME.6
mv $BUDIR$NAME.4 $BUDIR$NAME.5
mv $BUDIR$NAME.3 $BUDIR$NAME.4
mv $BUDIR$NAME.2 $BUDIR$NAME.3
mv $BUDIR$NAME.1 $BUDIR$NAME.2
cp -al $BUDIR$NAME.0 $BUDIR$NAME.1
NOW=$(date)
echo >> $BUDIR$NAME.log
echo >> $BUDIR$NAME.log
echo >> $BUDIR$NAME.log
echo "##### STARTING INCREMENTAL BACKUP AT $NOW" >> $BUDIR$NAME.log
echo >> $BUDIR$NAME.log
rsync --archive -L --delete $SRCDIR  $BUDIR$NAME.0/ 2>&1 | tee -a $BUDIR$NAME.log

You would create one of these scripts per directory you want to backup. BUDIR is the directory where you want to backup to be stored. This will probably be the same for every script if you have only one backup drive/server. SRCDIR is the directory you want to backup. NAME is the name you want the backup to be stored under. It can be a sub-directory if you want, so you can sort the backups into directories on the target.

Comment via email

SBWG 0.5.12

Entry created on 2020-09-27 Authors: steeph (337) Categories: Bash (30) Code (30) Computer (75) Linux (34) Projects (40) SBWG (17) Scripts (27) Software (50) Languages used: en (189) Topics: Projects → Code → Bash Scripts → SBWG (16) Projects → Web Sites (19)

Large leaps of progress are being made with SBWG. The feature set is growing satisfyingly rapid, some not so great ideas have been scrapped and as I'm learning more about Bash the code quality is also improving. I'm behind on writing a proper introduction, README and so on.

You can download version 0.6.17 here and I'll write something about it later.

See how large the leaps are? The version changed before I finished writing this entry.

Comment via email

SBWG - steeph's bash website generator

Entry Permalink (edited 2021-12-05) Authors: steeph (337) Categories: Bash (30) Code (30) Computer (75) Linux (34) Projects (40) SBWG (17) Scripts (27) Software (50) Languages used: en (189) Topics: Projects → Code → Bash Scripts → SBWG (16) Projects → Web Sites (19)

So in this entry I kind of just want to introduce SBWG a bit. Even though it's not the first time I post about the project here (it's Dec 2021 now) I realised there is no explanation on what I'm making and the page on the tag cat:SBWG consists pretty much only of updates. This is mainly a personal weblog and not a project web site. So a bit of background should be part of this introduction. To have the entry in the right place and order, I dated it back to Jan 2020, before I started to post updates about SBWG.

Every now an then I develop or intensify an existing interest in a topic and it becomes my main hobby. I've liked shell scripts ever since I started to use Linux. But I really started to write shell scripts for my own use properly in 2019. I wanted to write better and more rebust scripts. Learning a language or improving language skills works best by actually using the language. SBWG is my project that makes me use Bash to archive many different things that I previously didn't know how to accomplish.

SBWG (steeph's bash website generator) is a bash script that generates static HTML from simple text files. A tool that generates/updates this web site when it's run. Bash surely isn't the right choice of programming language for such a task. But I like Bash and don't really care. I try to make SBWG professional in all other respects. But it is a learning project. So there will probably always be things in the code that could be improved. The more features I implement and the more I learn the more I notice that I could have done better. But I do want to keep implementing new features (My list of ideas is long.) and not just rewriting what already works.

For information about the script, have a look at the project page. The README file in the downloadable package has extensive information about how it can be used.

Older version of this entry (obsolete and not actually interesting):

SBWG (steeph's bash website generator) is a bash script that generates static HTML from simple text files.

This entry does not have current information on SBWG. It was initially intended to hold an up-to-date introduction of SBWG with a download link. But I don't want to edit this blog entry all the time. So this stays obsolete and incomplete and I point you to the correct URL.

Even older version of this entry (obsolete and incomplete):

SBWG (steeph's bash website generator) is a bash script that generates static HTML from simple text files. I wrote it just for ~~fun~~ this website. And it's pronounced sibwig (because why not). It is not professional, not very adaptive to other website structures and does not have many features compared to most other static website generators. But if you know HTML and bash you can certainly adapt it for creating other websites. Or use it as it is, if you happen to want the same website structure as I have. If you want, you can download the latest version that I've published here .

The script creates a HTML structure from whatever text files it finds in the source directories.

The script generates an HTML file from every file in the directories pages/ and entries/, as well as blog-like listings for every tag found in these source files. Page files are generated into the html/ directory. Entry pages are generated into the html/entries/ directory. For every tag found in the entry source pages an additional HTML file is created in the html/tags/ directory. You can add more files to the html/ directory. As long as there is no file with the same name being generated or copied by the script, they won't be affected by the script. This way you can add more CSS files, media files, JavaScript files or anything else you might want to add to the website.

Folder Structure

html/ - This is where the script puts the resulting web site.

entries/ - Source files for blog posts are placed here. Each file is an entry.

galleries/ - Contains images that will be presented in a gellery view or attached to a blog entry.

pages/ - Source files for other HTML pages, that are not blog posts, are placed here. (e.g. front page, contact information, ...)

files/ - Can be used for downloadable or other files linked to from pages and entries.

tags/ - Working directory for generating an HTML file per tag. You may ignore it.

head - Header used for every generated HTML file.

header - Second part of the header. The script may put tags in between head and header.

sidebar - The sidebar that is generated and placed in every HTML file by the script. You may ignore this file.

footer - Footer used for every generated HTML file.

(More files, such as CSS and image files may be placed ~~here~~ anywhere and copied to their respective locations inside html/.

The html directory

This is where the script puts the resulting web site. It can be used as live webroot/symlink to a webroot or you copy your finished website from there to the webserver.

The entries directory

This directory is empty when you start a new website. You fill it with source files for entries/blog posts. An HTML page will be generated from each entry source files, all entries will be included in the all.html blog view and entries with tags will be added to the relevant tag pages. Entry source files start with an HTML comment that resembles a header that typically contains the creation date, a title and tags (categories, topics, langeage). All of these are optional. But the structure (the HTML comment) needs to be there in any case) See below for a detailed description of the source file headers with an example.

The pages directory

This directory contains source files for static pages that are not blog posts. An HTML page will be generated from each page source file. You can use it for a front page, contact page, etc. Page source files are similar to entry source files but they contain no tags. See below for a detailed description and an example.

The galleries directory

This directory can contain directory of images that will be generated into a gallery view. Each subdirectory will become a gallery. If a subdirectory (gallery) and an entry source file (blog post) have the same name, thumbnails of the images will be attached to the blog entry and a link to the gallery view will be placed at the top of the blog entry. A link to the entry will also be included on the gallery page. A gallery does not need to have a corrosponding entry though. You can simply create galleries by putting JPEG and PNG files into a directory and this directory into the galleries directory. Currently only files ending in .jpeg, .jpg or .png are recognised by the script.

The files directory

In this directory you can put files that should be available on the generated website but not be processed by the generator script. The content of this directory will be copied to html/files without any check. You can use this for any files that will be linked in pages and blog entries, image files that are not part of a gallery, JavaScript files, etc.

The tags directory and the sidebar file

This directory is used by the script to store information about tags found in entry files during the generation process. It is solely a working directory. Files stored in this directory will be deleted during a complete website generation. You can just ignore (er empty or delete) it. A future version of SBWG will likely switch to a better method of keeping track of the tags used and will not need this directory.

The sidebar file contains the last generated sidebar of the website. This file will be included in everey generated HTML file to enable navigating the website structure. You can ignore this file.

The head, header and footer

The head file contains the head of the HTML page structure. It will be used as the beginning of every generated HTML file When starting a new website it contains the neccessary things to generate a sample website. You can edit it to add meta tags and link or include style sheets.

The header file closes the HTML head and starts its body. It contains the header that will be included in every generated HTML file. When starting a new website it contains a logo and the name of the website. Feel free to include anything that you want to be on the top of every page of your website, like a menu, etc.

Having the head and the header files separate enables the script to add tags to the HTML head for individual pages. This is currently only used to add the page's title.

The footer file contains a footer that will be included at the end of every generated HTML file. Feel free to edit it to include anything that you want at the bottom of every page of your website. It also closes the HTML tag structure.

The root directory

More files can be placed at the top of the directory structure. As is the script copies the file style.css from here to the html directory. All other files (not mentioned above) will be ignored unless you edit the script to do something with files tht you have added.

Source File Format

Every source file (files in the directories pages/ and entries/) is supposed to start with at least three lines of HTML comment. Example source for a page file:

 This is the page content. It may consist of HTML or only text.

The first line can be used as an actual comment and is ignored by the script. The second line is used as the page title/entry title. For entries, any additional lines inside the HTML comments are used as tags/topics (one tag per line). The comment should be closed with a line only containing -->. Everything after that is the actual page/entry content. HTML can be used here and will not be filtered by the script. This may change in the future.

Example source for an entry file:

 <p>This is a blog post.</p>

Tag Icons

(tbd)

Dates

(tbd)

Options and Arguments

If the script is run without arguments it's using what I use regularly, which is options t, e, a, r and p. If arguments are given, the script will only generate the part that was selected with its corrosponding option. The following options exist:

-s Copy stylesheet files to the output directory.

-f Copy the files directory to the output directory.

-g Generate image galleries

-t Generate tag pages and the sidebar

-p Generate pages from the pages/ directory

-e Generate entry pages from the entries/ directory

-a Generate the all.html file

-r Generate RSS feed

-c A complete re-build. All of the above

Not all of these options are implemented, yet, but at this point, all except the RSS feed generation works well enough. Keep in mind though that this is all work in progress and I'm only meeting my own needs as I'm not aware of anybody else using this script.

Options that can be set through variables in the script

Not all available options can be set through command line arguments (yet?). There are a few variables that are being set at the beginning of the sctipt. Check out the first few lines. The comments above the lines where variables are being set should be sufficiant explanations of those variables. There you can set for example the size of gallery images and thumbnails, the number of blog post per page or the directory where the generated HTML site is placed to.

Major features still missing

RSS feed, nice layout and styles

Comment via email