Ken's picture

Hi everyone,

Few days ago i joined my site to Cloudflare .On Cloudflare, i set SSL/TLS encryption mode is Full (strict)

Today i got an email about cronjob daily

/etc/cron.daily/confconsole-dehydrated:
Certificate will expire
[2020-07-16 06:34:02] dehydrated-wrapper: INFO: started
[2020-07-16 06:34:02] dehydrated-wrapper: INFO: found apache2 listening on port 80
[2020-07-16 06:34:02] dehydrated-wrapper: INFO: stopping apache2
[2020-07-16 06:34:03] dehydrated-wrapper: INFO: running dehydrated
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:ietf:params:acme:error:unauthorized",
    "detail": "Invalid response from https://vidtubeworld.com/.well-known/acme-challenge/f_jS-NWkSIGaUECFDiEnMOBWq2HBlEPQEn3jwrlbqAc [2606:4700:3035::ac43:d353]: \"\u003c!DOCTYPE html\u003e\\n\u003c!--[if lt IE 7]\u003e \u003chtml class=\\\"no-js ie6 oldie\\\" lang=\\\"en-US\\\"\u003e \u003c![endif]--\u003e\\n\u003c!--[if IE 7]\u003e    \u003chtml class=\\\"no-js \"",
    "status": 403
  },
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/5911802422/6Di2Ig",
  "token": "f_jS-NWkSIGaUECFDiEnMOBWq2HBlEPQEn3jwrlbqAc",
  "validationRecord": [
    {
      "url": "http://vidtubeworld.com/.well-known/acme-challenge/f_jS-NWkSIGaUECFDiEnMOBWq2HBlEPQEn3jwrlbqAc",
      "hostname": "vidtubeworld.com",
      "port": "80",
      "addressesResolved": [
        "172.67.211.83",
        "104.31.71.53",
        "104.31.70.53",
        "2606:4700:3036::681f:4735",
        "2606:4700:3035::ac43:d353",
        "2606:4700:3030::681f:4635"
      ],
      "addressUsed": "2606:4700:3036::681f:4735"
    },
    {
      "url": "https://vidtubeworld.com/.well-known/acme-challenge/f_jS-NWkSIGaUECFDiEnMOBWq2HBlEPQEn3jwrlbqAc",
      "hostname": "vidtubeworld.com",
      "port": "443",
      "addressesResolved": [
        "172.67.211.83",
        "104.31.70.53",
        "104.31.71.53",
        "2606:4700:3035::ac43:d353",
        "2606:4700:3036::681f:4735",
        "2606:4700:3030::681f:4635"
      ],
      "addressUsed": "2606:4700:3035::ac43:d353"
    }
  ]
})
[2020-07-16 06:34:10] dehydrated-wrapper: FATAL: dehydrated exited with a non-zero exit code.
[2020-07-16 06:34:10] dehydrated-wrapper: WARNING: Python is still listening on port 80
[2020-07-16 06:34:10] dehydrated-wrapper: INFO: attempting to kill add-water server
[2020-07-16 06:34:10] dehydrated-wrapper: WARNING: Something went wrong, restoring original cert & key.
[2020-07-16 06:34:10] dehydrated-wrapper: INFO: starting apache2
[2020-07-16 06:34:11] dehydrated-wrapper: INFO: starting stunnel4
[2020-07-16 06:34:11] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.

Have someone any advice for this issue ?

Thank you guys

Forum: 
Jeremy Davis's picture

I'm not super familiar with use of Cloudflare (at least not pragmatically) myself. But FWIW Let's Encrypt themselves recommend using Cloudflare SSL certs (i.e. from Cloudflare CA) rather than Let's Encrypt.

Having said that, a little bit of googling suggests that it is quite possible to use Let's Encrypt certs with Cloudflare. But I don't think it lends itself nicely to usage of the automation via "HTTP-01" challenge method that we currently provide.

Essentially, the problems are two-fold. Firstly it appears that your current Cloudflare config is redirecting HTTP to HTTPS (and HTTP-01 challenge requires plain HTTP access). Secondly, it appears that the Cloudflare config is blocking Let's Encrypt servers from accessing the required "well known" URL to complete the challenge (returning a 403 error).

I suspect that there are ways that you can workaround both of these issues, but I'm not 100% sure of that. And even if you can, I suspect that there are better ways...

If you do wish to persevere with Let's Encrypt certs, most likely a better approach would be to use an alternate authorisation method. DNS-01 challenges probably lend themselves much more easily to this end. Instead of requiring your server to serve a specific token via a specific HTTP URL, a DNS-01 challenge requires you to set a specific token value for a specific DNS record (related to your domain).

For Let's Encrypt certs, we leverage software called Dehydrated (we use a custom wrapper with a custom hook script to respond to HTTP-01 challenges). Dehydrated also support DNS-01 challenges and there are a number of user contributed "DNS hook scripts" . The wiki notes 2 Cloudflare specific DNS hooks; a python one and a bash one. I can't vouch for either, but both appear to have relatively recent commits so probably both work.

If you go that way, you can use the version of Dehydrated that is pre-installed in TurnKey, but you won't want to use our wrapper script and you'll also need to configure ti to use the desired hook script (rather than ours). So you'll need to generate your own config file (/etc/dehydrated/conf) and change the cron job (or create a new one). You'll also need to do whatever config is required of your chosen hook (I assume that at least it will need some sort of API key or credentials to update the Cloudflare DNS).

Alternatively, you could go a completely different direction and disable our set up altogether and install an alternative Let's Encrypt client. E.g. "certbot" (apparently it has a "cloudflare DNS" plugin too).

Regardless of which way you go, I'd be interested to hear what you decide and how easy/hard/etc it was.

Richard van Dijk's picture

You can probably configure a page rule in Cloudflare to bypass the HTTP to HTTPS redirect for the challenge URLs. Then serve the challenge over HTTP on port 80 of your webserver. However, I agree with Jeremy that this is not the best solution.

I use Cloudflare Origin CA Certificates on my webservers that are behind Cloudflare, and that has always worked very well for me. More info: https://support.cloudflare.com/hc/en-us/articles/115000479507-Managing-C...

Ken's picture

thanks for your suggestion

i used the Cloudflare Origin CA Certificates on my webservers and it worked but on this server i have also mail service postfix and dovecot. When i access my mail in mail client , i got the error message show that this cert is not trusted. So i think this solution isn't the best in my case.

I choose the second solution using the DNS hooks bash script. When i run the ./dehydrated -c -t dns-01 -k 'hooks/cfhookbash/hook.sh' , i got another error message like this

=================================

Processing mail.vidtubeworld.com
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 1 authorizations URLs from the CA
 + Handling authorization for mail.vidtubeworld.com
 + 1 pending challenge(s)
 + Deploying challenge tokens...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   371    0   214  100   157   1491   1094 --:--:-- --:--:-- --:--:--  1496
+++ Wait for 10 seconds. +++
 + Responding to challenge for mail.vidtubeworld.com authorization...
 + Cleaning challenge tokens...
id: null
{"success":false,"errors":[{"code":7003,"message":"Could not route to \/zones\/dns_records\/null, perhaps your object identifier is invalid?"},{"code":7000,"message":"No route for that URI"}],"messages":[],"result":null} + Challenge validation has failed :(
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "dns-01",
  "status": "invalid",
  "error": {
    "type": "urn:ietf:params:acme:error:dns",
    "detail": "DNS problem: NXDOMAIN looking up TXT for _acme-challenge.mail.vidtubeworld.com - check that a DNS record exists for this domain",
    "status": 400
  },
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/6547262320/5WU_Qw",
  "token": "p_WvHMKzHhA2_u1_o2oO18RHsDiFUUSCTdGsJFGixJA"
})

================================

The problem is that i dont have the token to add into a TXT record for _acme-challenge.mail.vidtubeworld.com on Cloudflare DNS

Do you have any suggestion ?

Ken's picture

Hi everyone,

After checking some config files, i edited the config.sh file in cfhookbash folder 

case ${1} in
    "www.yourdomain.com")
        global_api_key="api"
        zones="zone id"
        email="email registered cloudflare account"
    ;;

    "yourdomain.com")
        global_api_key="api"
        zones="zone id"
        email="email registered cloudflare account"
    ;;

    "mail.yourdomain.com")
        global_api_key="api"
        zones="zone id"
        email="email registered cloudflare account"
    ;;

    # if you have another subdomains else, put them here also

    "sub.yourdomain.com")
        global_api_key="api"
        zones="zone id"
        email="email registered cloudflare account"
    ;;

....
esac

After that i run the command again and it worked for me :)

./dehydrated -c -t dns-01 -k 'hooks/cfhookbash/hook.sh'

Now i'm configuring the rest and i will update infor when i finish 

Thanks again :)

Jeremy Davis's picture

Great to hear that you worked it out and it's working for you now! :)

Thanks too for posting back to share your info. I'm sure someone else will find that invaluable!

Ken's picture

Finally i finish the task. I summarize here step by step the method i used, that may help someone else

Take the snapshot for container or vm that you do the test

1. Disable auto renew cert in confconsole

2. Setup

cd ~
git clone https://github.com/sineverba/cfhookbash.git

3.Configuration

  • Create a file domains.txt in the folder of dehydrated
  • Put inside a list (one for line) of domains that need certificates.

nano /etc/dehydrated/domains.txt

www.example.com
home.example.net
[...]
  • or use the old one 
cp /etc/dehydrated/confconsole.domains.txt /etc/dehydrated/domains.txt
  • Move to the folder of cfhookbash
cd cfhookbash/ 
  • Copy config.default.sh to config.sh
cp config.default.sh config.sh
  • Edit config.sh. To get values:
nano config.sh

case ${1} in

    "domain.com")
        global_api_key="key"
        zones="zone-id"
        email="email registered cloudflare"
    ;;

    "sub.domain.com")
        global_api_key="key"
        zones="zone-id"
        email="email registered cloudflare"
    ;;

    "sub2.domain.com")
        global_api_key="key"
        zones="zone-id"
        email="email registered cloudflare"
    ;;
esac

4. Usage

cd /etc/dehydrated
dehydrated --register --accept-terms

./dehydrated -c -t dns-01 -k '${PATH_WHERE_YOU_CLONED_CFHOOKBASH}/cfhookbash/hook.sh'


In my case

dehydrated -c -t dns-01 -k '/root/cfhookbash/hook.sh'

!!!!! Check carefully api-key, zone-id and email in the config.sh file, else you would got the error message

5. Post deploy

cd cfhookbash/
cp deploy.config.sh deploy.sh && rm deploy.config.sh && nano deploy.sh

case ${1} in

        "domain.com")
        #move certificate to private

        cp /var/lib/dehydrated/certs/domain.com/fullchain.pem /etc/ssl/private/cert.pem
        cp /var/lib/dehydrated/certs/domain.com/privkey.pem /etc/ssl/private/cert.key
        cp /var/lib/dehydrated/certs/domain.com/fullchain.pem /etc/ssl/certs/ca-certificates.crt

        #restart nginx
        #service nginx restart
    # for apache
    service apache2 restart
======================
the below part is not necessary for my case
        #exchange certificate for domoticz
        rm /home/sineverba/domoticz/server_cert.pem
        cat /var/www/domain/private/privkey.pem >> /home/sineverba/domoticz/server_cert.pem
        cat /var/www/domain/private/fullchain.pem >> /home/sineverba/domoticz/server_cert.pem
        cp /home/sineverba/domoticz/server_cert.pem /home/sineverba/domoticz/letsencrypt_server_cert.pem

        #restart domoticz
        /etc/init.d/domoticz.sh restart

    #restart mysensors
    service mysgw restart
========================
        ;;

esac

6. Cronjob

crontab -e
0 4 * * 1 cd /home/YOUR_USER/dehydrated && /home/YOUR_USER/dehydrated/dehydrated -c -t dns-01 -k '/home/YOUR_USER/dehydrated/hooks/cfhookbash/hook.sh' >> /home/YOUR_USER/cfhookbash.log

in my case

0 4 * * 1 cd /etc/dehydrated && /etc/dehydrated/dehydrated -c -t dns-01 -k '/path/to/folder/cfhookbash/hook.sh' >> /path/to/folder/cfhookbash/cfhookbash.log

7. Update / upgrade

  • Move to folder where you downloaded the hook bash
cd cfhookbash/
  • Type 
git checkout master && git pull

If service apache2 start fail, you should check the path to cert files and the following recommended apache config

https://community.letsencrypt.org/t/recommended-apache-config/58294

 

nano /etc/apache2/mods-enabled/ssl.conf

    #   Default certificate file
    SSLCertificateFile /path/to/cert/file/cert.pem
    SSLCertificateKeyFile /path/to/cert/file/cert.key

service apache2 restart

 

Because i installed also postfix and dovecot so i must re-check paths to the cert files (cert.pem, cert.key, ca-certification.crt) again on the following config files

/etc/dovecot/conf.d/10-ssl.conf

/etc/postfix/master.cf

/etc/postfix/main.cf

After that everything work fine 

 

!!!!! Attention

==========================

I got cert with the official url (https://acme-v02.api.letsencrypt.org/directory) and it working without error

The ACME URL for our ACME v2 staging environment is

https://acme-staging-v02.api.letsencrypt.org/directory

if you use this acme url for testing you will get the error message in mail client , show that the cert is not trust. After testing you can edit this url to official url in /path/to/folder/cfhookbash/config

CA="https://acme-v02.api.letsencrypt.org/directory"

then re-run

dehydrated -c -t dns-01 -k '/path/to/folder/cfhookbash/hook.sh'

==========================

I hope this instruction may help someone else

Jeremy Davis's picture

Thanks so much for posting back with your solution. I'm sure it will be of value to others.

Add new comment