You are here
So I am working on some customization. I have gone through a few steps:
Implemented a TKLDEV server
In GitHub to my account, forked from turnkeylinux-apps Core, TKLDEV
In GitHub to my account, forked from turnkeylinux BUILDTASKS
It looks like I want my "tweaks" to go into my (I will branch first) BUILDTASKS (no caps) repository
My question is, how do I get those tweaks to
A: Show up into my TKLDEV server for testing
and
B: Show up in "MAKE" based builds on subsequent TKLDEV builds.
Answering my question a bit,
a: my thinking is that I will need to set the buildtask folder on my TKLDEV server instance to point to my buildtasks repository
b: I will need to understand "which" buildtasks the TKLDEV overlay is pointing at and try to modify it.
I hope these questions are clearly stated.
thanks in advance.
Hey Landis! :) Answer to Q1
Awesome stuff mate! :)
Ok, so assuming that you are starting with a vanilla TKLDev, and aiming to produce your own custom Core appliance (built from 'core' repository buildcode) with a custom/tweaked buildtype (via 'buildtasks' repository). We'll leave TKLDev for now.
So to answer your first question:
Set up git in TKLDev and link to GitHub
Configure git and generate a new SSH key
First you need to set up git on your TKLDev. This only needs to be done once per new TKLDev. This is just example, you can use my example as is to get started, although ideally, adjust the bits in quotes - I suggest using the same email address as you have set in your GH account; then GitHub will show your account for any commits you push:
The next bit is to add the SSH keys from your TKLDev to your GitHub account:
SSH into your TKLDev (as root), then create a new keypair (adjust the email address; ideally it should be the same address as above/same address as registered with GitHub):
Allow it to use the default file (just hit enter). When it asks for a passphrase, you can add one if you wish, or leave it blank. Adding a passphrase increases security, but also increases inconvenience. If you add a passphrase; you'll need to enter it every time that you push or pull from GitHub - if you don't, you won't). So long as you only use this key for GitHub, then I personally think that no password is fine (just hit enter to have a blank password).
Link TKLDev to GitHub
Again, this part only needs to be done once. Once the key generation is complete (it may take a little while), then view the public key like this:
It should display a multi-line string that starts with 'ssh-rsa' and ends with the email address you used, with the middle part (~3 lines of wrapped text) consisting of a random string of characters.
Copy the whole output (including the 'ssh-rsa' at the start and your email at the end) then browse to the GitHub new SSH key page (you need to be logged into GitHub). Then in the top line, name your key (perhaps something like 'TKLDev01' - or whatever you want, make it something that can identify your TKLDev easily so you can revoke the key if need be). Then paste the full output of of your key in the "Key" box and hit "Add SSH key".
Link buildtasks code to GitHub
This bit needs to be done for each GitHub repo that you use. In your browser, browse to your buildtasks repo. Click the green "Clone or download" button. If it says "Clone with HTTPS", click the "Use SSH" text link. Then copy the url (FWIW in your case it's 'git@github.com:l-arnold/buildtasks.git'). Now back in the SSH session on your TKLDev, change to the buildtasks directory:
And now add your GitHub account as a "remote". Something like this (you can name it anything you like, I've used "my-github" in my example):
Then to check which remotes you have set up already:
TurnKey's repo should be listed there as "origin" by default (FWIW that is auto set up on firstboot of TKLDev). Convention is that your primary repo (that you can "push" to) is called "origin" and the original repo where the code comes from (i.e. TurnKey's in this case) is named "upstream". Although personally, I like to rename the TurnKey repo as "turnkey" so there is never any confusion (I rename my personal GH repo as "origin"). You can use whichever as you'd like (or change it to something else, or leave it as is). Moving forward, I'll use the terminology "origin" to refer to your repo and "turnkey" to refer to the TurnKey one. So to set that up:
If you run 'git remote -v' again, you should now see this:
Note that you can only use SSH key authentication (e.g. 'origin' in the above example) for repos that you have push access to (e.g. repos that you have forked into your own GitHub account). Otherwise use the HTTPS address.
So now you have git set up and your TKLDev is linked to GitHub (well at least the buildtasks repo is). If you wish to link your Core buildcode to your GitHub repo, follow the same steps above - in the "Link buildtasks code to GitHub" section - but do it within the 'products/core' directory and adjust the settings accordingly.
Adding tweaks and pushing to GitHub
Branches and getting the latest TurnKey code
So by default, you will be on the 'master' branch. Whenever you do development, you should work in a new branch. But before you do that, make sure that you have the latest TurnKey code. So in the 'buildtasks' directory, still on the master branch, run this:
That will pull in any new commits from the TurnKey buildtasks repo on GitHub. If this is a new TKLDev, then there likely won't be any.
Checking out a new branch
As noted above, when you do development, you should always do that in a new branch. Ideally the new branch is named something useful. For this example, I'll call it 'xen-tweaks'. Note that branch names should not contain any spaces and should generally be all lower case. To create a new branch:
And switch to it (i.e. check it out):
Hint: You can create a new branch and check it out all in the one command. To do that, can use checkout with the '-b' switch. So to do the above all in one command:
You can now push this (empty branch) back to GitHub, like this:
If you check your buildtasks repo on GitHub now, you'll see that you have a new branch called 'xen-tweaks' (although it will still be the same code as 'turnkey/master').
So now you're now in your very own branch, ready to hack on the code! :)
Committing your code
As you make changes, I suggest committing the changes as you go. Then you can see what changes have been done at different points and if things don't work, you can undo changes really easily.
You can tidy up the commit history later on, including things such as "squashing commits" (making 2 or more commits into one) and "rebasing" (interactive rebase allows you to reorder commits; often useful when planning to squash commits). I won't cover that now, but wanted to mention it. I think knowing that it can be tidied up later encourages you to commit often! :)
So to see what files you have edited, check the 'git status'. FWIW, TurnKey has a shortcut command which is 'gs'. If you haven't made any changes, you'll see:
If you change a file, you'll notice the filename noted in the git status, under "Changes not staged for commit" and any new files will appear under "Untracked files". It will include instructions on what to do to change the status of these files. As an example, I'm created a new file named "new-file" and edited "bt-iso" so 'git status' shows:
As noted in the commit message, to get these changes ready to commit, each file needs to be added:
Out of interest, those files are now considered "staged" (i.e. ready to be committed). Check the 'git status' again to see that:
Note that as mentioned in the git output above, you can instead use 'git commit -a', although, I would argue that's not a good habit to get into as it will only include files that already exist within the repo (i.e. "tracked" files - as noted in "Changes not staged for commit") and won't include the new files (i.e. "untracked" files).
Finally commit the changes (i.e. the changes that are "staged"):
By default it will open a text editor for you to write your commit message. Alternatively, you can use the '-m' switch to include a message:
If you want to push those changes to GitHub (e.g. perhaps you want some feedback from me?), then push as per above:
There is tons more cool stuff you can do with git, but that's probably enough to get you started! :)
Answer to Q2
If you read this in context of Q1, then hopefully this will make sense.
Bottom line is, that you just need to redo some of the steps from my response to Q1. So if you start from the steps in "Link buildtasks code to GitHub" from my answer to Q1, but instead of using the buildtasks repo, you need to use the core repo.
The same steps apply re setting up the remotes, pulling and pushing from/to Github, as well as creating new branches and committing code. Essentially the only things that will need changing are the remote urls and the branch name you choose, otherwise it should be pretty much the same.
Hopefully that all makes sense. If you have any issues or questions, please ask. :)
Awesome Jeremy
I will spend some time tomorrow. I did some of this before seeing your message but in a shortened form. I messed up my system I believe, by removing build tasks and re cloning them... My guess is zi knocked out the VMWARE helper program but I'm not sure.
I had a pretty succinct round of mini commits that I think will work.
Anyway, back to this tommorrow.
Thank you for your huge response.
Guessing I will need to ask how to install the VMware helper as my notes brushed over that more than it should have. It all did work a few weeks ago but now I want to get a few things hard wired rather than reverse engineered as I am prone to doing.
One thing at a time... :)
One word of advice - one that I could do to remember myself sometimes :) - just do one thing at a time...
I find that if I try to do too many things in one go, then even if it works as desired, I'm not actually sure which bit was the bit that made it work. Or whether it was a combo of things. Worse still, if it doesn't work, I can never be sure that I haven't introduced a new issue.
If I've learned nothing else while working with Alon and Liraz (and actually I've learned lots...) is that "slowing down" is often a great start to solving problems. Step-by-step methodical progress sometimes feel slower, but is usually actually faster. And when I finally do resolve the problem, I can be fairly sure of what the solution was (rather than having to guess).
Re what you noted:
Whilst just adding your fork as a "remote" is quicker and easier; if you did it right, then deleting and recloning shouldn't cause an issue. Essentially all 'tkldev-setup' does is clones buildtasks from GitHub. The only difference is that if you cloned it from your fork, then by default your repo will be the "origin" remote (and will "track" your repo, rather than the turnkey one).
TBH, I'm not sure why you think that the VMware tools may be the cause of your issues? Please feel to explain if you want some feedback.
Regardless, good luck with it all.
Well, where is /config/common.cfg ?
I will certainly need to go through your full instruction above Jeremy.
I have been able to pull my repository for a Joomla3 build
I also, likely made a mistake, and pulled my repository for buildtasks.
At present I am able to "make" a standards "product.iso" into the Joomla3 folder.
However, when I try to run any sort of BT command I get similar to:
If I run the same for XEN it will be line 76.
I had previously run a successful build on the LAMP appliance a few weeks ago, so I am assuming this is buggered up because of my Git Clone of my Buildtasks. Perhaps not.
OVFTOOL is installed properly.
Back with more news soon.
Found this in Optimized Builds (should get me going now)
https://github.com/turnkeylinux-apps/tkldev/blob/master/docs/advanced/optimized-builds.rst
Configuration & Setup
The default settings should be fine for builds you plan to use locally.
To keep TKLDev as slim as possible, we decided to not preinstall all of the dependencies. So please read through the Buildtask setup docs and install the appropriate dependencies (you won't need them all unless you intend to build all the appliance types).
Yep... got it going
I have a ./bt-iso joomla3 running now
Some small questions still.
Running on 2 servers just now ( I will get through your full set of instructions still. )
Glad you worked that out
I'm really glad to hear that you worked that out.
Perhaps we should create a readme file for buildtasks which gives the basic setup instructions and points to the TKLDev docs for further detail?!
Possibly one thing worth noting is that buildtasks is built and designed for our internal use (i.e. building all the different builds). However, we provide it publicly as there is clearly potential value for others, such as yourself. And it also allows us to get others to contribute improvements and/or bug fixes for the different build types.
Re your questions:
Firstly, it's probably worth noting that the behaviour of 'bt-iso' is a little different from all the other top level bt scripts (i.e. the other ones that start with 'bt-'). I'll explain 'bt-iso' in a little more detail below. But keep in mind that all the other 'bt-' scripts use an ISO that has been created by 'bt-iso'. If an appropriately named ISO does not exist, it will attempt to download one from the TurnKey mirror (or fail if it doesn't exist).
So 'bt-iso' will never download an ISO from the mirror - it will always attempt to build one (or fail). All the other scripts will check for a local ISO first, but will fallback to trying to download one.
If you run ./bt-iso, then it will build an ISO from the relevant product directory (or fail if it doesn't exist). E.g.:
Will build an ISO named "_NAME_&_VERSION_-stretch-amd64.iso" from the directory /turnkey/fab/products/joomla3. The "_NAME_&_VERSION_" will be the name and version noted at the top of the changelog. FWIW, the current top line of the joomla3 appliance changelog is:
So if you don't change it, it will build an ISO named "turnkey-joomla3-15.5-stretch-amd64.iso". You should leave the "turnkey" at the start, as that is used as a marker for later buildtasks scripts (so you'll hit issues later if you change that), but you can change the appliance name ('joomla3' in this case) and/or the version ('15.5' in this case). I suggest that you do change at least one of those so you won't get your own custom build confused with one produced by TurnKey. Perhaps the best way to go would be to add a "custom" (or similar) to the name? E.g. something like this:
Or, another example:
You can do that by adding a new changelog entry to the top of the changelog file. Please note that the new entry must be compliant with Debian changelog policy. Full details about changelogs - essentially our "changelog policy" can be found in the TKLDev docs.
When translated to the top line of a changelog entry, the above examples I gave, would look like this:
And this (respectively):
(Just add that same last bit as per these examples).
Note that the doc page is written explicitly for developers working on "official" TurnKey builds. E.g. someone developing a new product or maintaining an existing product intended to be built by us and released as an "official" TurnKey build. So if you're building a custom product for your own purposes, you don't need to explictly follow all the instructions there. So perhaps we should adjust the docs to differentiate between what "has to be done" (i.e. things won't work if you don't do this) vs "what we want done" (i.e. it will work, but it's not how we want official apps done).
Anyway, I hope that points you in the right direction.
No. Have a look at the bt-iso script (most of buildtasks is written in bash) and even if you don't understand much of it, you can hopefully get a gist of what it's doing. Essentially bash scripts are just lines of commands that you could run from the commandline!
So the core of what bt-iso does is (if the products/APP_NAME directory doesn't exist) a git clone of the relevant repo, or (if it does exist) a git pull (so if you change the branch you use, you may need to ensure that it is tracking a remote branch - otherwise you will get an error message).
Then it runs 'make clean, then make'. It then calls another buildtasks script ('bin/iso-release') to do a few additional steps; namely move 'product.iso' to /mnt/isos (or wherever "$BT_ISOS" points to) and rename it. It also generates the hash file (which you will need later if you build an alternate build).
As noted earlier, we use buildtasks internally to produce all our different builds. The 'publish' option is used by us to push the builds to an AWS S3 bucket (where some more steps occur such as verification and signing), before they are finally pushed to the mirror. If publish isn't used, then it shows that warning. But the warning is generally for our purposes (although if you configure all the AWS stuff in buildtasks, you could theoretically use publish if you wanted).
You can safely ignore that message when you are building manually for your own purposes.
That would almost certainly be the message from the 'git pull' step that I noted above.
I hope that helps give you an understanding of what is going on. And perhaps even assists you to understand a little more about bash?! :)
FWIW, if you want to debug your script (or just see what is going on in more detail) you can run it like this:
So working through.. I want to pull and work in "My Repo" only.
I have gotten down to the "tweaks" section and I see that on one of my servers I was indeed oriented at the Turnkeylinux repository (https).
I have setup my repository as Origin now but when I look at Branch I see only "master" still. I want to have my branch be the one I am looking at. On my original server it is "landis" (ie mine) and not "master".
Both Turnkeylinux and l-arnold do have a "master" branch so I'm not really sure which "master" is in use. I have made a few changes that I want to use.
I believe I should issue the command
or perhaps
Fundamentally, for now I just want to work in "origin landis" and not have any risk of writing to the turnkey system. As you saw, I did a fork and was able to control all this with GitHub desktop. Perhaps the easiest would be to simply RESET the GIT on Buildtasks, do a Git CLONE to my repository and move forward.
I did
which seemed to pull my 6 Commits from Branch "landis" into my local branch "master". I don't expect it wrote to my master repository yet. Anyway I will test a build now.
OK. More to do today.
Git will track 'master' from where it was cloned from.
Git will track 'master' from where it was cloned from. So in the case of a repo that you originally cloned from TurnKey, 'master' should always track the TurnKey repo. That's probably the way you want to keep it too (as then you can pull in changes that we make).
So to use your 'landis' branch, you just have to ensure that your git repo also has a 'landis' branch. The easiest way to do that is to push your local 'landis' branch to GitHub. Assuming that your 'origin' remote points to your GitHub fork - which is what I recommended and seems to be what you've done, you can do that like this (I've included some comments to explain what I'm doing so you can adjust as need be):
Now if you go to GitHub, you will see that your repo now has a new branch called 'landis" which should match the code that you have committed locally.
Some success, I still did not get Networking running
I added eth0 back into the list of Networking Interfaces but it seems I did not make it default on FirstBootInitHooks...
I got hung up on the close of that. Let me see where a forced boot takes me and whether I can run confconsole...
sorry not to answer your comment above (will do)
The build I was refering to around the networking was definitely on My l-arnold/joomla3 b- landis where I had made the changes the day before.
It was on a second server that I was following your instructions and had the branches a bit more intertwined. The code I was shooting for should be in the "xen-tweak" your term (direct modification of the "xen patch" on my side.
To Confirm current STATUS
My XEN build did Start up in Linode. Unfortunately my XEN TWEAK did not actually get ETH0 (is it ETH1) enabled and Networking is Not Enabled. This is where I was a while back when I was doing this from the Mirror Repos.
I am wanting to attach a Screenshot from ConfConsole which is running on my GLISH but not seeing how here on the forum.
I will see what I can find (you told me once) what I can do to get ETH0 (or ETH1) enabled manually. I do have access to SSH.
Getting there. Happy I could boot. I did get the "auto reset" of ROOT password to not happen in FIRSTBOOTINITHOOK, so YAY there.
sorry not to answer your comment above (will do)
I really need to know how to "splice" the cloud networking in
From what I can garner, I need the networking setup "ala" the Cloud Patch and the build setup otherwise (bzip2) ala the XEN patch.
I will try to differentiate the changes that each of these make to the standard ISO build. Is there an easy way?
Info about Networking
Likely something in the XEN Networking setup itsel. Here are a few Shell pieces of info
/etc/init.d/networking restart
Restarting networking (via systemctl): networking.serviceJob for networking.service failed because the control process exited with error code.
See "systemctl status networking.service" and "journalctl -xe" for details.
failed!
----
root@joomla3 /# systemctl status networking.service
* networking.service - Raise network interfaces
Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor prese
Active: failed (Result: exit-code) since Wed 2019-10-23 04:14:46 UTC; 52s ago
Docs: man:interfaces(5)
Process: 1567 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=
Process: 1563 ExecStartPre=/bin/sh -c [ "$CONFIGURE_INTERFACES" != "no" ] && [
Main PID: 1567 (code=exited, status=1/FAILURE)
Oct 23 04:14:46 joomla3 systemd[1]: Starting Raise network interfaces...
Oct 23 04:14:46 joomla3 ifup[1567]: udhcpc: SIOCGIFINDEX: No such device
Oct 23 04:14:46 joomla3 ifup[1567]: ifup: failed to bring up eth0
Oct 23 04:14:46 joomla3 ifup[1567]: udhcpc: SIOCGIFINDEX: No such device
Oct 23 04:14:46 joomla3 ifup[1567]: ifup: failed to bring up eth1
Oct 23 04:14:46 joomla3 systemd[1]: networking.service: Main process exited, cod
Oct 23 04:14:46 joomla3 systemd[1]: Failed to start Raise network interfaces.
Oct 23 04:14:46 joomla3 systemd[1]: networking.service: Unit entered failed stat
Oct 23 04:14:46 joomla3 systemd[1]: networking.service: Failed with result 'exit
lines 1-17/17 (END)
It looks like you aren't getting eth0 & eth1 interfaces
My first thought is that perhaps you are getting the newer, so called "predictable interface names". We have disabled that functionality via a common conf script which adds 'net.ifnames=0' to the kernel boot parameters that are loaded by grub (the bootloader). But perhaps for some reason that isn't working as expected on Linode?
Perhaps it's worth trying one of the other possible workarounds?
https://github.com/turnkeylinux/inithooks
Maybe the issue is here.... changed last year.
https://github.com/turnkeylinux/inithooks
So.. I need to dissable XEN in my XEN
I should probably stop commenting.
Inithooks/run
reads like this. Particular attention to the XEN Service at the bottom.
I need to disable that. I am not using XEN. I am only using XEN's TAR packaging (and also removing root password set)
--------
Moreover
HOW do I disable SYSTEMCTL and alow traditional function of TurnkeyLinux settings in CONFCONSOLE etc?
It looks like I need to add more to my XEN Patch changes.
Sorry for my string of comments there Jeremy
Sorry for my string of comments there Jeremy
It is all interesting to work through.
---
I see this in my TKLDEV log. Pretty sure I would do well turning off the XEN service and probably also the LXC service:
The service file to use is set in buildtasks
Those multiple service files are to cater to the differences in the environments the different builds are to run on. They are set within the buildtasks 'bin/build-tag script. If you wish to use a different one, simply change which file you touch there. It's possibly worth noting that whilst we explicitly create a 'default' file if the default service file is ok, the only ones that are checked for are lxc, xen & openstack.
If you look at each of the service files, whilst they are all enabled, they have conditions which depend on which file exists in the /var/lib/turnkey-info/inithooks.service/ directory. Note that the ISO build does not have that directory by default. E.g. on a local LXC appliance I have running (gitea):
Which should mean that the (default) inithooks.service is disabled (because it fails the "test"). Let's check:
Yep! Note the "start condition failed". :) Let's have a look at those conditions:
So it's not running because of the failed condition: "ConditionPathExists=!/var/lib/turnkey-info/inithooks.service/lxc". (the '!' at the start is a logic condition which means "not", so that file must not exist for the service to start. Let's check the 'inithooks-lxc.service' status:
So whilst that is not running ('Active: inactive (dead)'), you can see from the log entries that it has run fairly recently. FWIW inithooks runs at every boot, but only gives interactive dialog on first boot (unless of course it's pre-seeded). Let's double check the conditions for the 'inithooks-lxc.service' starting:
So that path condition line doesn't have the leading '!', so the path must exist (which it does in my case) for the service to start...
I hope that explanation gives you some insight into how those inithook files are decided...
Worked through (Predictable Naming issue)
OK. We are in business.
Need to Run the following command (and therefore also get in my TKLDEV build)
Needed to set
to have ETH0 Be allowed. Need to also make it Default
Will Need to Clean the ConfConsole List of Interfaces.
I closed another thread from a few months ago on the same issue:
https://www.turnkeylinux.org/comment/reply/17985
Thank you Jeremy for your inputs!.
Ah dear... I should have read the whole thread... :)
It looks like you were way ahead of me! :) Although good to see that my guess about predictable interface names was right...
I wonder why the kernel option that we use isn't being picked up by Linode's KVM implementation? Perhaps it's something to do with the fact that you are using the Xen build on KVM? FWIW I use KVM lots myself (although I install directly from the TurnKey ISO) and it works fine for me?! Anyway, I'm really glad that you worked it out. Perhaps we should include that somewhere as well as (or perhaps instead of) the kernel command line entry?
As another FYI, the OpenStack build we now provide (which results in a .qcow2 disk image) is targeted at KVM backed OpenStack (.qcow2 in the native KVM disk image type, like .vmdk is the native VMware disk image type). It should (at least in theory) work on a vanilla KVM too (although TBH, I've never tested, but probably should some time...). Whether or not that format is supported via upload to Linode or not though is a completely separate matter.
So if you need a tarball of the OS (i.e. somewhat like the format that our Xen build provides, but for KVM), then you might be best off developing your very own custom bt script?! The "headless" and "cloud" patches should still apply, but likely much of the rest you could pull in selections from the OpenStack build (much of that likely) and the Xen build (the final packup into a tar ball).
Regardless, it sounds like you've had good progress. Well done mate! :)
If you need a hand developing your custom script and/or some code review, let me know. I probably won't have tons of time until I can get the v16.0RC of Core released (really soon I hope) but am happy to add my 2c. Just push your changed branch to your GitHub repo and give me a ping.
In direct reply (plus as always, a big Thank you Jeremy!)
Agree that some of these issues are that I was basing the work on the XEN patch process. That comes from the fact that I can move the build pretty easily in the BZ2 package and still control the fundamental aspects of Drive Size, Memory, Processor etc easily. I have tried to extract the Qcow2 as well as most other builds. I have been able to install from ISO but the install gives me an environment that is not scalable up and down as needed. Nor can I make an quick/installable image from it.
But, it is likely that I will want to use some of those general patch processes and not use the XEN packaging. I just ran a search on XEN on my installed server and there were way more files there than I expected.
Best case scenario I will get a better splice between TKL and Linode's Kernel and also get a few requisite Linode files as part into the Linode Patch (like Network Helper).
Several variables for sure (understatement).
I just ran a New ISO and running a new XEN with most (but the last 3 changes). Will see how it works.
Thank you again Jeremy!
2 small items I am not clear how to do to update my BuildTasks "
I've updated my XEN Patch directory (as a testing) base and made pretty much the same thing in a LINODE Patch directory.
I thought I would first test in the XEN directory. If all works, then Try in the Linode Directory. If that works, I would revert my XEN directory back to default and try to work moving forward in my Linode directory.
BUT, I need 2 things.
1: How can I apply the command (or similar to) in SED type Patch Lingo
2: How can I "at least" Rename the xen (inithooks service
from:
to:
or potentially, How do I not create that file. I see that lxc is not created so there is something that created the inithooks.service/xen file (which in turn stops normal usage of netstat and confconsole)
OK 1 more
3: How do I make sure that eth0 is the default NIC ? It may already be. I've seen some code forcing eth1 elsewhere but is commented out in buildtasks (ec2 patch perhaps)
That should do it.
Thank you in advance. Landis
Answers
1
Why not just use that line as it is in the patch conf script?! As per other conf scripts (e.g. in common and appliance build code), patch conf scripts are bash scripts. So including that command within the patch conf should "just work" as it is.
FWIW sed is a (sort of) acronym for stream editor. In other words you feed data (text) in (i.e a stream of data), it modifies it (edits it) and then spits out the modified result. The '-i' switch allows to to modify the contents of a file in place. Behind the scenes, it moves the original file, streams the contents, edits it and writes it to a new file (with the original name) and deletes the original (renamed) file.
As such, sed is only useful for modifying the contents of text files, not for creating symlinks as you're doing with 'ln -s'. FWIW 'ln' makes a link, the '-s' switch makes it a softlink/symlink - as opposed to a hard link; which is what 'ln' does by default).
2
The short (and hacky) answer is to just delete (or rename) the file in the bt script (e.g. bt-xen) after the bin/build-tag script has been called, but before the root filesystem is tarred up. E.g. insert this new line (currently line 88):
Or to rename, use 'mv' (note that renaming/moving it has no real value as it's just an empty file and unless it's named 'xen', 'lxc' or 'openstack', it ignored and the default inithooks.service will run):
The longer (and proper) answer:
Change the line that calls the bin/build-tag script from this:
to this:
Then modify the bin/build-tag script case statement to include linode in the appropriate place. E.g. if the default inithooks.service is ok, change the appropriate line from:
to:
If you want to use one of the other ones, just insert '|linode' where appropriate. E.g. if you wanted to use the OpenStack one (I suspect that is likely the one you want?), then change:
to:It will then create the appropriate file in '/var/lib/turnkey-info/inithooks.service/' so the desired service file is the one that's used.
FWIW the '|' (pipe) character (or in some contexts, a double pipe - '||') is often used in bash as a logic OR operator - as in the case statements.
3
It should be by default. I.e. the network/interfaces file you posted earlier should do the trick. If the default interfaces file is different in the Xen build, then either find where that is happening and remove the code and/or overlay file.
IIRC the default interfaces file has eth1 commented out (i.e '#' character at the start of the line). The comments ('#') are removed from eth1 (thus enabling eth1 as well) in some of the patches
Or alternatively, you could try adding it to the patch overlay. Although please note that if it's being changed in the conf script, that will still likely be changed as the conf script runs after the overlay is applied. A dirty workaround for that would be to put it in the overlay under a different name (e.g. 'overlay/etc/network/interfaces.new') and move that over the top of the Xen specific one at the end of the Xen patch conf script (or even in bt-xen after the patch and before the tar command).
Having said that, it's always better to prune unrequired code than add additional code (and therefore additional complexity) which essentially has no benefit.
You also noted that kernel commandline option that we use (i.e. 'net.ifnames=0') isn't working and you had to use an alternate method to disable the so called "predictable interface names" (so they revert to the legacy ethX naming convention). IIRC that was #1 (above) right? If so then you've already covered that in #1! :)
Unless of course, you mean in confconsole? If that's the case, then again, it should be using eth0 by default and if it's not, then I suspect that there is something in the buildtasks bt-xen script and/or xen patch that is changing it.
Thanks Jeremy
1: I will put that into CONF. Maybe I even have but I think I did not know where.
2: I will do the "hacky" approach first on the XEN patch then the more proper change on the Linode patch for the rename or remove component. The Rename is needed to get control of the Networking within the XEN context that gets offloaded to the Predictable Names component otherwise. The rationale here is that the main program throws a switch if it finds a file there called XEN. I don't think it even has any content.
I will definitely study the "good" way to do it and also work to understand and use the various build structures.
On 3 plus (towards the bottom of your comment... directly above)
The part about the "netifnames=0" I don't actually recall but it might be a more elegant solution to the sym link. All of this of course needs a cross-linked Glossary to understand it (sarcasm/joke perhaps).
continued.. Yes, I do need to also edit "confconsole" and ideally just show a few Network Adapters (eth0, maybe eth1, perhaps something else). I can see the value in some situations for having 2 Network adapters. For instance, if one is building a Database connection over perhaps a VPN or SSH Connection to allow Data to flow on a resultant localhost (I do that quite a bit actually). For now I want to just focus on where to edit the ConfConsole list of network adapters (or if all is working, perhaps I will disable the whole Network Config part of ConfConsole -- something that gets done here and there in the patches).
Have a great day. I will work on this a bit more tonight. My second build completed without errors. I need to throw these few switches manually and I will test to see if it is working otherwise.
O have put these changes in and see I need to fix one thing. Got the error:
For now I just commented the line in bt-xen and will try the rest.
--- Update --
Huge Improvement still. SYSTEM is running and I am able to login both to back end and to Joomla. Webmin, Websehell all good.
I cannot login to Adminer just now, not sure why.
I will attempt to move my documentation of this over to my own repository for a while.
Glad for progress on this. Need to let it sink in a bit and start trying some of a few new patches.
Thank you.
Ok.. TKLDEV from github/turnkeylinux-apps vs from /turnkeylinux
-- perhaps solved - see last line. Still a question in the middle
My repositories get a little confused if I keep TKL's naming conventions.
I have a repo forked from https://github.com/turnkeylinux-apps/tkldev
I try to run a Make from the folder and I get
I can clear the directory and pull straight from turnkeylinux-apps but I know I have the same in my fork.
Thoughts on this one?
-- Just trying straight to ./bt-iso tkldev (seems to be working) Wonder why?
Hmm, not sure why?!
TBH, that's a bit of a weird one (especially when it works ok via ./bt-iso tkldev?!
IIRC you can get more detailed info by setting "DEBUG=y" before you run 'make'; either by exporting it first:
Or on the same line:
Assuming that you are building from your own custom branch, please feel free to share your changes and I can have a quick look. Alternatively, commit your code and push your branch back to GitHub.
Add new comment