draft0 - a shared blog by just some people

Wow, has it been more than a year again without publishing a new SBWG version? Another year where I didn't have much energy for things besides work. But I finally worked on it again. It's still evolving a lot. But the list of things I still want to do is slowly becoming shorter.

SBWG 0.13.0 is a pretty stable version again. I've tested it with several sites for a while and I'm now also using it to generate this web site. You can get it from there.

Apart from the almost permanently ongoing task of cleaning up the code formatting line by line because I started out with a mess of tabs and spaces mixed in different circumstances, I find the following changes worth describing here.

File attachments: Image files are now only embedded as a thumbnail and displayed as a gallery if the file type is one of the following: image/avif, image/bmp, image/gif, image/heic, image/jp2, image/jpeg, image/jpm, image/jpx, image/jxr, image/png, image/svg+xml, image/tiff, image/vnd.microsoft.icon, image/webp or image/x-jp2-codestream. Even though not all of them are supported by most web browsers and there are browsers that support formats not on the list, those are the image types that I find widely supported enough to try to display their contents instead of only linking to the original files. If a file attachment is in a subdirectory, the same sub-path is now used on the generated web site, allowing for several files of the same name to be attached to different entries. That is something that is now possible because file attachment now not necessarily need to be named after the entry they are attached to. Their file name does not matter if the directory they are in is named after the entry. For axample: ENTRYNAME, ENTRYNAME-files or ENTRYNAME-images

Since SBWG sometimes changes how certain features work, it is possible (as in thinkable. It's not like SBWG is used by so many sites that there has been an actual example of it.) that a web site created for one version of SBWG is not generating as expected with a newer version of the script. That is why it is now possible to declare in a site's settings file that it may not be generated by a SBWG script newer than a certain version. If your site uses hooks it may be a good idea to use this new feature so that you get a message when trying to use a wrong version of SBWG instead of noticing after a possibly long generation process that everything does not look right. To activate this restriction, simply add a < sign followed by the latest version allowed to the first line of the site's settings file. For example: #SBWG <0.13.0

Accessibility as an afterthought: For some reason I didn't many things wrong from the start when it comes to accessibility. I've always felt that design decisions are and should be subject to my personality and mood. This is why SBWG is only written for Bash (so far) and so many specialty features are the way they are. I guess I've let this policy spill out on decisions like how readable generated sites are. I've never been good with designing web pages entirely on my own. I'm better at implementing CSS than at design decisions. But this goes so far that I've created an image gallery feature (for image file attachments) with modal preview without using any JavaScript. To realise that I had to break some HTML rules and create HTML that is not viewable without a basic minimum of styling (to hide some elements). I am very very slowly drifting off that course of not caring about things like that. As a start, I've changed the layout of web sites generated with SBWG to contain their menu after the content. For a styled site there's no difference (e.g. using the default "elth" style). But in text browsers (and hopefully also screen readers) this means the entire menu with all topics and the whole tagcloud isn't displayed/read before getting to the actual content of a page. To make it easy to access the menu, there is now a "Jump to the menu" link at the top of every page. Another improvement in this regard is inclusion of image descriptions as alt attributes where descriptions are found in the meta information embedded in the image files. SBWG now looks for all sorts of tags that might hold a useful description of an image, or at least a title or keywords. Fields meant for image descriptions are preferred over others that probably only hold inferior information. This should in some cases already generate useful alt attributes. But I know that I'm going to have to actually write those image descriptions for images used on my site. It's a start. Like I said. I'm moving very very slowly on this.

Style set handling: The most recent large change has been the rewrite of everything related to fetching and embedding style sets. I made some minor compromises but simplified some parts a lot. Changes emerging from the re-write include: Style sets can be placed in different directories now. SBWG looks for the requested style sets in the web site's own styles directory as before. If it isn't there, it continues looking in the user's home directory (`~/.local/share/sbwg/styles/`), then in a global styles directory (`/usr/local/share/sbwg/styles/`), then in the styles directory relative to the script file (in case SBWG is used as a portable directory, not installed). This way styles can be provided globally, e.g. with an install script, additional or altered styles can be installed by a user for their own sites and additional or altered styles again can be limited to a single web site. And lastly a stylesheet file can be chosen by path no matter where it's localed. But this file will be its own style set. Special use case, I know. But if you happen to have a use for that feature, there you go. Style files named reset.css are no longer treated specially. It wasn't a requested feature and I never used it myself. If you use alternate styles on your site (which is unlikely enough) and there are style definitions you want to be included in all style sets, simply include them in all style sets. If you want to refrain from using redundant CSS, put that code in a separate file like before with reset.css) and create links for each style set. The result will be same as before but more self explanatory. When using alternate styles on a web site (so the user can select from them in the browser), all non-styling files from all style sets are now being used. Again this is more in line with what you would expect. Although I might change this again later because the way it used to be was more practical. There were good reasons for it. The basic minimum of styling that should be applied to a site generated by SBWG is now hardcoded. If no style whatsoever is requested, a file with those minimal rules is generated and linked on generated HTML files.

Other than that it's mostly code improvements, HTML cleanliness improvements, formatting improvements and UI/style improvements of a tiny nature. But I'm glad I have two of the larger todos - attachment changes and the style rewrite - (almost) done.

I'm glad to have this version done and published. Even though it has new features compared to the last published version, I've used it enough to feel relatively comfortable saying it is also more stable and has fewer bugs than the last version. There are known bugs still left. But those aren't new.

Caching options and settings are slowly getting a bit elaborate, but also close to what I imagined them being able to do. There are new cachegroups and parts of entries can now be chosed as individual cachegroups, giving the user more control over what is cached and what caches are used. The lifetime of caches can now be set to make sure that no matter how the caching options are used, no outdated content, tags, attachments, etc. will stay on the web site for too long because cache files have been forgotten to be purges. The cache lifetime can be chosen from 1 second to unfinite. The directory used for persistant caches can now be chosen via command line option. I think the caching options are now in a state in which they can seriously be used reliably. Changes to existing cachegroups, like in previous versions, will probably not be made anymore. Just more cachegroups will be added.

The README file has grown a lot. Not only from new features and options. It's now also more complete. A lot of bugs have been fixed. And some general little code quality improvements took place. The stylesheets of the example web site have been improved a bit.

The messaging and logging system has been completely rewritten. It's not really an important part of the script and strictly doesn't matter for its functionality at all. But I decided to have a messaging system and a logging option that doesn't rely on shell redirects. So I did want to redo it properly. Different message types can now be redirected using file descriptors. The option parser has also been party re-written. It's approaching a state in which it's close to what I think personally a complete option parser for Bash should be like.

Alternate styles can now be defined in a web site's settings file. That way alternate style sheets will be offered to the web browser. If the browser supports alternate style sheets, the visitor can select from them to view the page in a different style. There is now an option to change the tagstyle, enabling a web site to have tags of the same type grouped for a more readable look of the entries' headers. Tags now show in parenthesis the number of entries that carry that tag. The number of file attachments is also displayed below entries.

The options for generating a single entry, page or gallery that is not located in the input directory/isn't actually part of the web site can now be used. They were very buggy or nonexistant before. Entries (and other items) with duplicate names are handled in defined ways now. Text files can now be used as file attachments to entries. But if the script determines one to look like a SBWG source file, it is omitted/not displayed/not linked to.

There are still big plans for future versions. Small and big features as well as a complete re-thinking of how the input directory is structured. Nicer looking web sites, more flexible command line options, more options for speeding up the script in cases where a complete website is updated/re-built. A pause like the one before this release will probably occur more often in the future.

It's been two months since I published a new version of SBWG (and therefore since I worked enough on it to make me feel that an increase in the version number may be justified). I hardly get around to working on it, lately. But when I do, I try to make some solid small steps towards the improvements I wish do be done to SBWG.

So, now there is a new version (0.10.11) but I haven't published it yet because before I do I want to test it a bit more when I'm not as fucking tired as I am right now. Writing this entry could actually be considered part of the testing that I still want to do. But as I said, I'm fucking tired right now. So I'll continue this entry another time.

Edit 2022-09-12: I did some tests with fake content offline and feel confident that I fixed more than I broke in this version. I'll add it to the project page soon.

Two long procrastinate areas that have been overdue to get fixed for a while now are permanent caching and parallelisation. For permanent caching, the bugs that I found are fixed. Aggressive caching with option `--cache` (`-C`) works and certain groups can be selected (excluding others) for permanent caching. In my test this speeds up everyday web site re-generations by up to 100%. But it also means the user has to be aware of the cache files that persist after SBWG is done and will be used the next time. Changes to existing entries will not take effect without removing the relevant cache files. For parallelisation, the bugs that I found and that affect the generated HTML are fixed. I still consider parallel mode to be experimental. Sometimes messages on stdout are weird or cut off. But who uses very verbose mode and looks at every message anyway? Parallelisation is functional and usable is what I mean to say. In my test on a Core i5 with 4 cores (8 threads) it speeds up everyday web site re-generation by around 380% with default settings. It depends a lot on how much of web site generation is generating thumbnails and other image sizes of file attachments or gallery images. Those aren't sped up that much because the CPU already is the bottleneck for those. HTML file generation profits more from having more parallel threads than CPU cores (or threads) running.