You are here
John Carver - Thu, 2014/01/09 - 01:44
Discussion of DevOps including continuous integration, continuous delivery, IT automation, and orchestration using the TurnKey Linux platform. This discussion was moved here from the LXC blog post, http://www.turnkeylinux.org/blog/announcing-turnkey-lxc.
Forum:
Jenkins - Continuous Integration
Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on building/testing software projects continuously (e.g., like CruiseControl) and monitoring executions of externally-run jobs.
Information is free, knowledge is acquired, but wisdom is earned.
Ansible - An IT Automation Tool
Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.
Information is free, knowledge is acquired, but wisdom is earned.
Ansible 1.8.2
I've just pushed the rc3 version of the Ansible appliance to https://github.com/Dude4Linux/ansible. It is running the current stable release of Ansible, version 1.8.2.
There is still some more development work to be done before final release. The ToDo list includes:
Information is free, knowledge is acquired, but wisdom is earned.
rc4 release
I've just pushed the rc4 version of the Ansible appliance. The first versions of the appliance used apt-get to install some of the python packages required by Ansible, but used pip to install Ansible itself. This led to a situation with a mix of older (from apt) and newer (from pip) packages. The result was a lot of warning messages when trying to upgrade the python packages. Version rc3 was an attempt to avoid this by installing everything using apt, but that led to other problems. The latest version, rc4, bootstraps the installation of pip, and then uses pip to install Ansible and all of it's dependencies. I'm hoping this will be the easiest to keep updated.
Information is free, knowledge is acquired, but wisdom is earned.
Great work John
Sounds like a well thought out and tested solution. Obviously software from the repos is preferred, but sometimes it's not the best option. With your testing and rationale I see no reason why your suggested way to go wouldn't be the best option...
Best practice for sharing ISO's?
Thanks Jeremy. Since most people won't want to take the trouble to setup TKLdev just to build an ISO for testing, I'm looking for a place to upload and store the image files. SourceForge has been revised and I'm not sure my account there is still available. I think GitHub still frowns on uploading ISO file there. What is the current best practice for sharing image files for testing?
Information is free, knowledge is acquired, but wisdom is earned.
I wasn't aware that SourceForge may be an issue...
Late last year (or perhaps even early this year) I logged in again (it's been quite a long time...) and all seemed to be as per usual. I enable another user upload rights to upload an ISO there which he did and all seemed to go ok. So unless something has happening in the last month or so I'd still think that's the best option.
If SourceForge is definitely no longer any good, then perhaps we need to look at a TurnKey "Community" upload/download (or better still build) facility. That would be ideal really... If appliances could be designed and tested by community members then just built by a cloud hosted TKLDev which spits out all build types...
Not sure when the change occurred.
I'm not sure when the change occurred. It's been over a year since I last visited. If I go to http://sourceforge.net/account/login.php, I get page not found.
I just figured out that I need to create a new project, in order to get the login screen. I could go ahead and create a project and upload the iso there, but I like your idea of a "Community" shared upload facility better. My TKLdev setup can only build iso's and I would really like to test other formats, i.e. LXC. Would have to figure out how to upload a build package rather than an iso if your suggestion is to work. Maybe a tar-zip of the project directory excluding the build directory? Or maybe the files could be pulled from the developer's github account. Submit your project's url, wait a few minutes, and then download the result in whatever format you want. Just a thought.
Information is free, knowledge is acquired, but wisdom is earned.
Yeah SF have changed stuff around a bit
But it looks like you are still there! :) As is the TurnKey community downloads page...
You already had upload rights, but I've just added you to the admins as well.
And FWIW the SF login page is now: https://sourceforge.net/auth/
As for the TKLDev output; currently ISO is all that TKLDev will output. I have been pushing Liraz to get the other build code public ASAP. It's currently all tangled up with some sensitive stuff that can't be public. The build code is part of the automated program that turns the ISOs that came out of TKLDev into all the other build formats and uploads direct to the TurnKey mirror network. Once the code that builds the other formats is public then people will be able to produce any format they desire. It will make it easier for new formats to be supported (or current formats to be improved) too. I have also suggested that the (TKLDev) build process stop at an earlier generic point (rather than building an ISO by default) as IMO making an ISO then converting that to other formats seems a bit redundant (especially OVZ and LXC which are really just archived root directories...). Anyway we'll need to wait for all that...
Guess what!?!?
I'm super excited as the guys have released the 'buildtasks' publicly. Its the code that builds all the different build formats... TBH I have no idea how it works (or how you apply it) but I'll be having a look really soon. If you beat me to it, please feel free to post back anything you glean (up to you but if you'd rather a new thread than taking this one further away from it's intention please do so...)
Oops almost forgot the link: https://github.com/turnkeylinux/buildtasks
Uploaded turnkey-ansible-13.1-rc4-wheezy-amd64.iso
Thanks Jeremy, I had forgotten I had upload rights there. I've uploaded a copy of the latest ISO file into a new folder, turnkey-ansible-test. http://sourceforge.net/projects/turnkeylinuxcom/files/turnkey-ansible-test/turnkey-ansible-13.1-rc4-wheezy-amd64.iso/download
Hopefully, other interested parties will give it a try, kick the tires, and provide some feedback. I'll post a quick getting started guide below.
Information is free, knowledge is acquired, but wisdom is earned.
Updating packages not installed by apt
As I pondered the pros and cons of installing packages via apt-get versus git or pip, I realized that while using either git or pip for installation provides access to the latest stable releases, it also shifts the burden of managing updates and upgrades to the TKLdev developer. As I'm writing this, it occurred to me that there is also the matter of security, which we tend to forget since it has long been handled by the apt package distribution system (repositories and keys).
I took a stab at the problem of managing updates/upgrades by writing a wrapper script for /usr/lib/inithooks/firstboot.d/95secupdates. To allow for appliance specific updates, I added a directory, /etc/turnkey/update.d/ where appliance specific scripts can be placed. Note: it does not yet distinguish security updates.
/usr/bin/turnkey-update
/etc/turnkey/update.d/ansible
Information is free, knowledge is acquired, but wisdom is earned.
Nice one John...
I'll give Alon and/or Liraz a nudge to have a look at this and see what they think. I think that the idea is sound but I will defer to them on implementation, etc.
QuickStart Guide to the TurnKey/GNU Ansible appliance
Download the latest release ISO from SourceForge Turnkey Linux Community (here). Install the ISO in your favorite virtual machine, VMware, VirtualBox, ProxMox, or whatever. Note: The virtual machine must be capable of installing from an ISO, so OpenVZ, LXC, etc are not yet supported. During the install process you will be prompted to enter passphrases for root and ansible.
The first time you login as ansible you will be prompted by keychain to enter the ansible password again. This should only happen the first time you login after a reboot. Keychain is starting the ssh-agent and storing the passphrase so Ansible scripts don't keep prompting you for the passphrase. I could not figure out how to make it seemless without entering the password twice. On subsequent logins, keychain will find the running ssh-agent.
Check that ansible is up and running.
Add more hosts
Edit ansible's inventory file, currently /etc/ansible/hosts, and add some additional hosts for testing. I recommend adding only disposable test machines until you are very comfortable with how ansible operates. For now, you will have to manually create and install the test machines. Later I hope to automate the creation of test machines using LXC. I'm also assuming that all test machines are TurnKey/GNU appliances.
Examples for hosts and groups can be found in /etc/ansible/hosts.example. We will create a group named 'test' and add a few test hosts. Hosts can be identified by IP address or by name. Host names must be registered either in /etc/hosts or in DNS. You must be able to ssh to each of the test hosts and login as root. I recommend you do this now for each test host so it's entered into the known_hosts file. I'm assuming that all of the root passwords are the same, otherwise you'll have to run the following step 6. once for each host. My inventory file now looks like this:
Create init playbook
In the ansible home directory, make a directory for ansible playbooks and copy the following yaml script into ~/playbooks/init.yml
Init the test hosts
In order to work it's magic, ansible needs root access to your test hosts. The init playbook will first make sure that python and python-apt packages are installed. It will then create an admins group and grant it's members passwordless sudo access. Next it creates an ansible user, makes it a member of the admins group, and adds it's own ssh key to the ~/.ssh/authorized_keys file.
Warning: Before adding any production hosts to the inventory file, make sure you have taken precautions to secure the Ansible appliance and properly protected access to both the root and ansible accounts.
Run ansible-playbook to execute the init.yml playbook. The '-k' option prompts for a password. At the prompt, enter the root password of the 'test' hosts, not the password for the ansible user. The '-l' option limits the action to the 'test' group defined in the inventory file because the init.yml playbook specifies 'all' hosts, otherwise ansible would try to initialize itself.
Note that most of the tasks report "ok:". This was because they had been completed by an earlier run on a previous version of the ansible server. The only thing that needed to change was the installation of the ssh public key from the ansible account of the freshly installed server.
Run adhoc commands
Now that the ssh key has been installed, we should be able to run some adhoc commands using ansible.
Experiment with Ansible Galaxy
Now you may want to experiment with ansible-galaxy to download a role or create a new one. Browse Ansible Galaxy to find an interesting role to download e.g. lxc
Make your own role.
Enjoy
I hope this QuickStart Guide has been helpful.
Information is free, knowledge is acquired, but wisdom is earned.
Ansible 14.0 released
The Ansible TurnKey GNU/Linux appliance has been released as part of the 14.0 stable suite of appliances.
Information is free, knowledge is acquired, but wisdom is earned.
Add new comment