I assume you will use the TKLX Core? (So, using the Omnibus Postgresql install and not installing one ourselves.)
Until I dig in and start work on it, I'm not 100% sure exactly how it will go. As far as teh buildcode goes, it won't be a rebase back on Core (it will be a continuation of the current GitLab buildcode repo). But as all appliances are essentially based on Core anyway, it sort of will be based on Core. Sorry, that probably makes limited sense...
In an effort to clarify my last paragraph; the new build use all the contents of the GitLab Omnibus package. E.g. it will include git and Postgres from the Omnibus package (rather than from Debian repos). I think that answers your question better!
If you feel you would like some configuration changes to the Omnibus installed components (NGINX and POSTGRESQL) you might consider doing post-install configuration & reboot script so we can more easily reproduce an out-of-the-box GitLab equivalent.
The way we would generally do that, is that any (new) defaults would be provided via an overlay or (preferably) tweaked via a conf script. But I anticipate that any changes we make would only be changes to GitLab defaults, not radical rewrites or anything. So working on the assumption that it would be nothing particularly significant (i.e. stuff that a "reasonable" somewhat advanced end user may do), I wouldn't expect any changes that we make would have any significance over and above what GitLab would come across with any end user installed system. Thus should not have any impact on any bug reports and/or support via GitLab.
Having said that, assuming that the Omnibus package is built in a sane way (and the defaults are included in the package as text files, then copied into place at install time), documenting how to get the original defaults from the Omnibus package should be pretty straight forward.
The main benefit I am hoping to gain from this exercise, if possible, is a working / optimised TKLBAM. I have not yet sorted out TKLBAM to do it myself. I expect that a base image on the TKLX server is required to really benefit from TKLBAM's incremental backup architecture?
Yes, we will be creating a new TKLBAM profile for the updated appliance (as we do anyway for each release). Moving to the Omnibus install will have the advantage that it should actually work a bit better than the current GitLab TKLBAM profile.
Based on experience to date, post-omnibus-installation configurations relevant to the TKLX VM admin console / wizard are as follows:
1) Configure GitLab settings to support AWS as external data-store for large data items
2) Confiure GitLab settings for Mattermost connectivity
3) Configure SSL Certificates
4) Configure a RAM-disk swapfile so GitLab does not trigger the dreaded OOM-killer resulting in 500 errors.
Hmmm, I'm not sure that we would be configuring too many of those things, initially at least. We could consider adding them as confconsole plugins though. Are these initial options within some sort of GitLab "install/first-run wizard" or something? TBH, I've never seen mention of most of those before...
Re specifics:
1. You mean like git-lfs right? AFAIK git-lfs should already be an option OOTB?! And other than perhaps dependencies (boto?) , I would imagine that it would be relatively minor config required. If you have further info/insight to share in that regard, please do so.
2. Again, isn't that something that comes OOTB? AFAIK the Omnibus package includes Mattermost support already? (Although, perhaps disabled by default?) At least, that's what my recent research lead me to believe...
3. As our implementation is webserver agnostic, I wouldn't expect this to be an issue. If you want a common cert for GitLab, Webmin and Webshell, then you could use the Confconsole plugin we provide. Otherwise if GitLab has some other implementation built in, then using that should not cause any issues. I would potentially expect one to override the other, but I wouldn't expect that to be problematic, although I guess we'll need to see how GitLab do it. And obviously it'll need testing.
4. In my testing (admittedly nothing compared to your first hand experience) a larger server with more RAM is a far superior option on AWS than adding swap. It gives a far superior user experience (RAM is tons faster than any harddrive, even SSD) and as AWS charge for IO (beyond the threshold that I forget OTTOMH), if swap is being used heavily, you may not even be saving any money!
OTOH I guess if you are trying to squeeze every last bit out of a server that is really too small for it's peek usage, then it might be a good thing. I am somewhat hesitant to set that up by default, although again a confconsole plugin could be an option? Having said that, I do note that GitLab recommend 2GB swap, even when using (the recommended) 8GB min RAM so perhaps we should consider it...?! (I think I prefer my lightening fast Gitea server which uses about 256MB RAM! :-p) FWIW all the reading and reviews I've seen recommend a t2.medium as a minimum for GitLab on a single AWS EC2 instance (even for a single developer). Again I'd be interested in your experience.
I'm hoping to get onto this ASAP. Once I have something I am happy to share it, and would love some feedback on it. Although I won't have an AMI, only an ISO initially.
Not 100% sure how it will go...
Until I dig in and start work on it, I'm not 100% sure exactly how it will go. As far as teh buildcode goes, it won't be a rebase back on Core (it will be a continuation of the current GitLab buildcode repo). But as all appliances are essentially based on Core anyway, it sort of will be based on Core. Sorry, that probably makes limited sense...
In an effort to clarify my last paragraph; the new build use all the contents of the GitLab Omnibus package. E.g. it will include git and Postgres from the Omnibus package (rather than from Debian repos). I think that answers your question better!
The way we would generally do that, is that any (new) defaults would be provided via an overlay or (preferably) tweaked via a conf script. But I anticipate that any changes we make would only be changes to GitLab defaults, not radical rewrites or anything. So working on the assumption that it would be nothing particularly significant (i.e. stuff that a "reasonable" somewhat advanced end user may do), I wouldn't expect any changes that we make would have any significance over and above what GitLab would come across with any end user installed system. Thus should not have any impact on any bug reports and/or support via GitLab.
Having said that, assuming that the Omnibus package is built in a sane way (and the defaults are included in the package as text files, then copied into place at install time), documenting how to get the original defaults from the Omnibus package should be pretty straight forward.
Yes, we will be creating a new TKLBAM profile for the updated appliance (as we do anyway for each release). Moving to the Omnibus install will have the advantage that it should actually work a bit better than the current GitLab TKLBAM profile.
Hmmm, I'm not sure that we would be configuring too many of those things, initially at least. We could consider adding them as confconsole plugins though. Are these initial options within some sort of GitLab "install/first-run wizard" or something? TBH, I've never seen mention of most of those before...
Re specifics:
1. You mean like git-lfs right? AFAIK git-lfs should already be an option OOTB?! And other than perhaps dependencies (boto?) , I would imagine that it would be relatively minor config required. If you have further info/insight to share in that regard, please do so.
2. Again, isn't that something that comes OOTB? AFAIK the Omnibus package includes Mattermost support already? (Although, perhaps disabled by default?) At least, that's what my recent research lead me to believe...
3. As our implementation is webserver agnostic, I wouldn't expect this to be an issue. If you want a common cert for GitLab, Webmin and Webshell, then you could use the Confconsole plugin we provide. Otherwise if GitLab has some other implementation built in, then using that should not cause any issues. I would potentially expect one to override the other, but I wouldn't expect that to be problematic, although I guess we'll need to see how GitLab do it. And obviously it'll need testing.
4. In my testing (admittedly nothing compared to your first hand experience) a larger server with more RAM is a far superior option on AWS than adding swap. It gives a far superior user experience (RAM is tons faster than any harddrive, even SSD) and as AWS charge for IO (beyond the threshold that I forget OTTOMH), if swap is being used heavily, you may not even be saving any money!
OTOH I guess if you are trying to squeeze every last bit out of a server that is really too small for it's peek usage, then it might be a good thing. I am somewhat hesitant to set that up by default, although again a confconsole plugin could be an option? Having said that, I do note that GitLab recommend 2GB swap, even when using (the recommended) 8GB min RAM so perhaps we should consider it...?! (I think I prefer my lightening fast Gitea server which uses about 256MB RAM! :-p) FWIW all the reading and reviews I've seen recommend a t2.medium as a minimum for GitLab on a single AWS EC2 instance (even for a single developer). Again I'd be interested in your experience.
I'm hoping to get onto this ASAP. Once I have something I am happy to share it, and would love some feedback on it. Although I won't have an AMI, only an ISO initially.