Core & TKLDev v17.0 Release Candidates - ready for battletesting

Update 2022-04-13: Please see the v17.0 stable release announcement for details about the stable v17.0 Core & TKLDev releases.

Drum roll please... I'm proud to announce the pre-release of the shiny new major version of TurnKey GNU/Linux. The first v17.0 Release Candidates of Core and TKLDev are ready for download, inspection and open for feedback..

Core and TKLDev

As per always, we are pre-releasing our base OS appliance, Core v17.0rc1 and asking the community to give it a spin and let us know what you think. As has become convention, we are also pre-releasing our "build tool in a box", TKLDev v17.0rc1. For the uninitiated, TKLDev is literally the mother of all appliances! It what we use internally to develop, build and run preliminary tests on new appliances.

If you have a particular appliance you are interested in, then building it to ISO and taking it for a test drive would be an awesome experience and a huge help to us and the rest of the community. If you need more specifics on that, please ask and I may even spend some time writing up a blog post on what's required and where things might hiccup (during development). Otherwise, please feel free to browse the TKLDev docs and/or ask questions (here in the comments, or open a new thread on the forums).

Download the RCs and help us test them

Core (64bit / amd64 build)342MB ISO  ( changeloghash filemanifest )

TKLDev (64bit / amd64 build)373MB ISO  ( changeloghash filemanifest )

Quick overview of significant changes

You can view further details of specific changes/bugfixes via the relevant changelogs (links above) and I'll do a full outline of the changes when we do the final stable release of v17.0. But in the meantime, here's a quick overview of major changes:

  • Rebase on Debian 11/Bullseye.
  • Updated ISO build mechanism to support UEFI boot. Note that the installer still doesn't (yet) support UEFI install, but it's the first piece of the puzzle.
  • Minor improvements and bugfixes for TurnKey Python3 based custom software (ported to Python3 for v16.0), including Confconsole and Inithooks (and their dependencies). Wherever possible, TurnKey custom software continues to depend on default Debian packages (rather than our own tools/forks).
  • Webmin - Update to latest v1.982, with latest stable version of the 'Authentic' theme. (Note there has since been an update to v1.983; which I hope to also package for the v17.0 stable release).
  • Key rotation of individual keys for specific purposes (i.e. new key for images, and one new key for each TurnKey apt repository; 'bullseye', 'bullseye-security' & 'bullseye-testing').
  • Update, refactoring and port to python3 for remaining build tools and dependencies, primarily Fab (the appliance building tool/mechanism) and Pool (our package building infrastructure tool/app).
  • New caching proxy config (Squid) that supports caching HTTP & HTTPS files now built into TKLDev.

  • This release continues our work to bring the TurnKey base more into line with vanilla Debian, whilst still providing ready to use appliances, our special toolset and secret sauce (well not really secret, because you can view the open source code anytime on GitHub). The idea behind this is that by default our appliances should "just work", but are much more approachable by non-seasoned TurnKey veterans with some Debian/Ubuntu experience.

    Expected timeframe for v17.0 stable release appliances

    As per always, there is still plenty to be done including quite a few feature requests that I'd love to see included in v17.0. On the other hand, I don't really want to hold up the release of v17.0 stable appliances any longer than I have to, so at least some of that will almost certainly be delayed until later in the lifetime of v17.x. Some things, such as UEFI install, still require some work, so whether or not they'll make it in remains to be seen. Regardless, we'll try to get as much in as we can whilst still pushing v17.0 out the door ASAP. We release "when ready" so I can't give any firm commitment on which appliances will be released when. We will however seek to prioritise our base appliances (Core, LAMP, LAPP, NodeJS, etc) and I hope to have a batch of shiny new releases ready for publishing (as the first batch) early in the new year.

    Please help test and give us your feedback

    As with previous "point-oh" releases, this is a major OS transition. As such, we need plenty of testing! We've already done a fair bit of testing in house, but the more the merrier! So hopefully you get a chance to give the RCs a test drive. We welcome all feedback, especially constructive critique. Please post below, open a new thread in the forums (requires a free website user account) or bugs can be reported directly to the Issue tracker (free GitHub user account required).

    If you have any spare cycles and hope to help out, we'd especially love to hear from you. Please reach out if you aren't sure where to start or need some guidance. Look forward to hearing from you! :)

    Comments

    IanH's picture

    Created a TKLDev VM in Proxmox. initially confused that there was already a "core" folder under "products". Deleted that and used git to get the core files.

    Make seemed to run ok until right at the end it gives an error. (see below)

    So recreated VM and tried again using the core folder already in TKLDev. Same error.

    Last lines of compilation are shown below. 

    I can try any ideas you suggest. TKLDev VM is ready to roll.

     

    cp build/root.sandbox/usr/lib/syslinux/modules/bios/vesamenu.c32 build/cdroot/isolinux
    cp build/root.sandbox/usr/lib/syslinux/modules/bios/gfxboot.c32 build/cdroot/isolinux
    cp build/root.sandbox/boot/vmlinuz-5.10.0-11-amd64 build/cdroot/live/vmlinuz
    cp build/root.sandbox/boot/initrd.img-5.10.0-11-amd64 build/cdroot/live/initrd.gz
    rm -f build/cdroot/live/20sandbox.squashfs
    xorriso -as mkisofs -o build/product.iso -r -J -V core -b isolinux/isolinux.bin -c isolinux/boot.cat -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e efi.img -no-emul-boot -isohybrid-gpt-basdat build/cdroot/
    xorriso 1.5.2 : RockRidge filesystem manipulator, libburnia project.
    
    Drive current: -outdev 'stdio:build/product.iso'
    Media current: stdio file, overwriteable
    Media status : is blank
    Media summary: 0 sessions, 0 data blocks, 0 data, 12.1g free
    xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
    Added to ISO image: directory '/'='/turnkey/fab/products/core/build/cdroot'
    xorriso : UPDATE :      16 files added in 1 seconds
    xorriso : UPDATE :      16 files added in 1 seconds
    xorriso : NOTE : Copying to System Area: 432 bytes from file '/usr/lib/ISOLINUX/isohdpfx.bin'
    xorriso : FAILURE : Cannot find path '/efi.img' in loaded ISO image
    xorriso : NOTE : -return_with SORRY 32 triggered by problem severity FAILURE
    make: *** [/usr/share/fab/product.mk:516: build/product.iso] Error 32
    

     

    Jeremy Davis's picture

    Firstly, thanks tons for the heads up, not to mention testing TKLDev out.

    The TL;DR of it is that I have wound back some significant changes to the cdroots. The code in the cdroots repo provides the base files for the ISO, including menu, boot options, etc. I had hoped to implement UEFI boot support in v17.0, but we really need to finalise v17.0 so we can do a stable release.

    I have pushed an updated version of fab that should resolve the issue.

    Please install it like this:

    apt update && apt install -y fab
    IanH's picture

    Same error with VMWare ESXi.

    Perhaps I am totally misunderstanding the instructions (although they are not difficult) or there is some subtlety missing. Trying to compile LAMP does not get fare before a python mysqldb (?) requirement is not found.

    TKLCore downloaded works fine so some disconnect as to how you do it and the instructions, Let me know what I can try next.

     

    Jeremy Davis's picture

    As for the LAMP error, it wouldn't surprise me that the packages might need adjustment for v17.0. It hasn't yet been updated (the common code for LAMP based appliances has, but not the LAMP appliance itself). Having said that, I did build it the other day, so I probably have uncommitted changes that can be pushed.

    Hopefully I should get to that within the next few days... I'll try to have a look ASAP and post back.

    IanH's picture

    Your fix allowed the Core ISO to be built.

     

    Following up about the lamp error (you did say try out your favourite VMs!)

    This is the error that stops compilation of lamp fairly early on:

     

    Package python-mysqldb is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another source
    
    E: Package 'python-mysqldb' has no installation candidate
    Traceback (most recent call last):
      File "/usr/bin/fab-install", line 567, in <module>
        cmd_install(
      File "/usr/bin/fab-install", line 232, in cmd_install
        installer.install(packages, ignore_errors)
      File "/usr/lib/python3.9/dist-packages/fablib/installer.py", line 364, in install
        self._install(packages, ignore_errors)
      File "/usr/lib/python3.9/dist-packages/fablib/installer.py", line 214, in _install
        raise Error(
    fablib.installer.Error: Errors encountered installing packages
    make: *** [/usr/share/fab/product.mk:569: build/stamps/root.build] Error 1
    
    Jeremy Davis's picture

    So first up, as you've likely guessed, the error is because it can't find the 'python-mysqldb' package. The packages are declared in the plan. As a short term measure, and assuming that it isn't a hard requirement (which in the case of 'python-mysqldb'; it isn't) you could just comment it out. Note that the plan uses C/Makefile type syntax, so like PHP syntax, a double slash ('//') prefix will make it ignore the rest of the line, thus skipping trying to install that package. FYI (unlike PHP), '#' can be used for inline comments - although not at the start of a line; inline comments can also be surrounded by slash, asterisk (same as PHP), i.e.: '/* ...comment here... */')

    As further context, in Debian Bullseye/11 (basis of v17.x) python2 has been (mostly) removed. Generally packages that are/were named 'python-...' are python2 packages; mostly python3 counterparts exist and are named 'python3-...' instead.

    So in the case of 'python-mysqldb', the package that we want is 'python3-mysqldb'. Meaning that the "proper" fix for the error, is to update the relevant line to:

    python3-mysqldb          # A Python interface to MySQL

    (The comment is optional, but may as well leave it there - so it's clear what the package does and why its included.)

    [update] I just quickly checked and it looks like that's all you should need to do. I just built LAMP after adjusting that package name.

    IanH's picture

    Thanks Jeremy for your help on these issues.

     

    I did indeed recognise that change to Python3 was causing the problem but had no idea how to correct it.

    Updating that line in plan/main AND updating "fab" allowed the compilation to complete. I did test commenting out the line in plan/main but that failed.

    Anyway I now have a version of LAMP to test with various photo databases to check that all is well.

    Will report back as I test.

     

    Ian

     

    IanH's picture

    I knew this was not likely to succeed but hopefully the following will be of use.

    wordpress/plan/main needs to be amended as follows.

    Initial errors showed tinymce and php-gettext were responsible. Modified plan as below:

    #include <turnkey/base>
    #include <turnkey/lamp>
    
    libjs-cropper       /* wordpress dep */
    libjs-jquery        /* wordpress dep */
    libjs-prototype     /* wordpress dep */
    libjs-scriptaculous /* wordpress dep */
    libphp-phpmailer    /* wordpress dep */
    libphp-snoopy       /* wordpress dep */
    php-php-gettext         /* wordpress dep */
    php-gd              /* wordpress dep */
    //tinymce             /* wordpress dep */
    php-zip             /* wordpress recommends */
    php-imagick         /* wordpress recommends */
    
    php-curl            /* needed by some plugins */
    
    unzip               /* most wordpress plugins and themes are zip archives */
    

    "tinymce" had no target. https://tracker.debian.org/pkg/tinymce seems to indicate it is no longer in Debian.

     

    Compilation then proceeded further  until this error:

    rm build/root.patched/etc/resolvconf/resolv.conf.d/original
    rm build/root.patched/etc/apt/trusted.gpg~
    warning: file or directory 'build/root.patched/etc/apt/trusted.gpg~' not found!
    rm build/root.patched/etc/apt/trusted.gpg.d/debian.gpg~
    warning: file or directory 'build/root.patched/etc/apt/trusted.gpg.d/debian.gpg~' not found!
    rm build/root.patched/etc/apt/trusted.gpg.d/turnkey.gpg~
    warning: file or directory 'build/root.patched/etc/apt/trusted.gpg.d/turnkey.gpg~' not found!
    rm build/root.patched/var/log/dpkg.log
    rm build/root.patched/var/log/bootstrap.log
    rm build/root.patched/var/log/apt/term.log
    rm build/root.patched/var/log/apt/history.log
    rm -r build/root.patched/var/www/html
    # apply the product-local units
    # apply the product-local root overlay
    if [ -d overlay ]; then fab-apply-overlay overlay build/root.patched; fi
    # run the product-local configuration scripts
    $(call run-conf-scripts, conf.d)
    fab-chroot build/root.patched --script conf.d/main
    + WPROOT=/var/www/wordpress
    + NAME=wordpress
    + USER=www-data
    + DB_PREFIX=wp_
    ++ mcookie
    + DB_PASS=145707b3bb6946d70e4decd6edcdd7dc
    + ADMIN_USER=admin
    + ADMIN_PASS=turnkey
    + ADMIN_MAIL=admin@example.com
    + PHPINI=/etc/php/7.3/apache2/php.ini
    + sed -i 's|^memory_limit.*|memory_limit = 128M|' /etc/php/7.3/apache2/php.ini
    sed: can't read /etc/php/7.3/apache2/php.ini: No such file or directory
    make: *** [/usr/share/fab/product.mk:569: build/stamps/root.patched] Error 2

    Searched for relevant file for PHPINI and found wordpress/conf.d/main where I changed "7.3" to "7.4".

    Compilation completed and produce product.iso!!!

    Testing it I have found there is no request for a WordPress password during install. I do have a password and its encrypted form so modified in adminer and could then log in to WordPress.

    Looking at plugins, there is no TinyMCE, just a replacement for it called "Advanced Editor Tools (previously TinyMCE Advanced)". Installed in WP and works.

    At the end of installation there is minor confusion over the unmount/eject commands - just a detail for tidying up.

    Once the vital absence of WP password is fixed, this looks good to go.

    Hope all that helps. Almost gave up a few time but in the end it was a bit of fun!

    Ian

    Jeremy Davis's picture

    Great work. Thanks Ian! That's helpful.

    FWIW, I am not surprised on the lack of admin password on firstboot. We refactored the inithooks software a little, so the wordpress.py python script needs updating. It's pretty easy to do, but only when you know how... If you are interested, this is the change that's needed (as shown on a different appliance).

    Next thing, we just need to teach you how to use git and GitHub so that you can just open a pull request with the required changes! :)

    Pages

    Add new comment