Tag: SSH

  • SSH. Part Duo.

    Securing Linux SSH with Duo two-factor authentication.

    A few days ago, I was chatting with someone about authentication. They mentioned Cisco Duo. I’ve used Duo before and knew it as an Enterprise-grade Identity/MFA provider. But I did not realize that Duo offered an “Enterprise. At home” tier of service with their “Duo Free” edition. In this post we’ll revisit the securing SSH post and will use Duo for our 2-Factor Authentication.

    Duo offers multiple editions based on the number of users that the installation will support and the features available. For Duo Free, the limit is 10 users. That should be fine with our use case.

    Duo provides a ton of excellent and easy to follow documentation instructions. They integrate with many products and make configuration easy. The integration that we’re interested in, for this post, is Duo Unix, documentation is found here.

    Set up

    Before we begin, we need to do some pre-requisite ground work. Unlike my other SSH post, we’ll be using a RHEL derivative AlmaLinux instead of Debian. I won’t bore you with the installation steps for AlmaLinux, but assume that it is version 10, minimal install.

    And, of course, you need to create an account with Duo. Again, it’s pretty simple, so I trust you can do it.

    Let’s configure Duo side first as we’ll need some configuration parameters for our integration.

    Duo

    If you don’t need the 30-day trial Duo offers you, you can switch to Duo Free. To do so, go to Billing -> Billing on the left, scroll down and select “Duo Free” for “Edition” under the “Manage Subscription” section. Accept Terms and Conditions and click the blue “Update Subscription” button.

    Next, in Application -> Application Catalog, we’ll search for “Unix Application” and will add it to our environment.

    In the new screen, we’ll name our application, select “Enable for all users”, make any changes for the phone greeting. This screen can be found in Applications -> Applications. In the “UNIX Application” screen you’ll see the “Integration key”, Secret key”, and “API hostname”. We’ll use these values when we set integration up on our Linux server.

    Linux SSH

    Now, we need to configure our Linux server. First, because we have a minimal installation, we need to install the required packages. Because we’re using a RHEL derivative (AlmaLinux), we’ll use dnf. We need to install gcc as we’ll be compiling the duo_pam module from source.

    sudo dnf install wget openssl-devel pam-devel selinux-policy-devel bzip2 tar gcc nano -y

    Now, download and extract the source code. Make sure you’re using the actual names of the .tar.gz file as it’ll change as the new version is released.

    wget --content-disposition https://dl.duosecurity.com/duo_unix-latest.tar.gz
    tar zxf duo_unix-2.2.1.tar.gz
    cd duo_unix-2.2.1

    Now, compile and install the Duo PAM module.

    ./configure --with-pam --prefix=/usr && make && sudo make install

    After the installation, let’s take care of SELinux, if you have it in Enforcing mode. Make sure to run these commands in the duo_unix-2.2.1 folder, the folder where we extracted the source code to.

    sudo make -C pam_duo semodule
    sudo make -C pam_duo semodule-install
    sudo semodule -l | grep duo 

    The last command should return a line authlogin_duo, indicating the SELinux module is installed.

    It’s time to configure Duo to allow for 2-factor authentication. Edit the pam_duo.conf file by running sudo nano /etc/duo/pam_duo.conf. Use the screen in Figure 4 to get the ikey (Integration key), skey (Secret key), and host (API hostname) values. Uncomment pushinfo = yes by removing a ; and add autopush=yes and prompts = 1. This will enable push notification to a phone so all you need to do is click a green “Approve” check mark button (but you need to do so on the first try).

    Now, we’ll configure SSH and PAM. First, we’ll configure SSH to use password authentication + Duo 2FA for SSH. sudo will NOT require 2FA. Run sudo nano /etc/ssh/sshd_config and make the following changes to the SSH config (add the line AuthenticationMethods as it doesn’t exist by default):

    PubkeyAuthentication no
    PasswordAuthentication no
    UsePAM yes
    KbdInteractiveAuthentication yes
    UseDNS no
    AuthenticationMethods keyboard-interactive

    Next, edit with sudo nano /etc/ssh/sshd_config.d/50-redhat.conf and comment out the line ChallengeResponseAuthentication no with a #.

    Edit the SSH PAM configuration with sudo nano /etc/pam.d/sshd and make sure the auth section is as follows:

    auth	   required     pam_sepermit.so
    auth	   required pam_env.so
    auth	   requisite    pam_unix.so nullok try_first_pass
    auth	   sufficient /lib64/security/pam_duo.so
    auth	   required pam_deny.so

    Now we can reboot, cross our fingers, and try to log back in.

    We should be greeted with a normal SSH login. But after putting the correct password in, we’ll get a Please enroll at https://<looong sting>. Copy the URL to your browser and follow the directions. Then try log in again and you should receive a Push notification on your phone. (I had a small issue after enrolling into Duo with sshd complaining about exceeded LoginGraceTime. Restarting sshd with sudo systemctl restart sshd solved this issue).

    Public Key Authentication

    To use public keys together with Duo, we need to make some modifications to our configuration. First, in /etc/ssh/sshd_config change PubkeyAuthentication no to PubkeyAuthentication yes and AuthenticationMethods keyboard-interactive to AuthenticationMethods publickey,keyboard-interactive.

    In /etc/pam.d/sshd comment out the line auth requisite pam_unix.so nullok try_first_pass. This way when you log into your SSH server using the public key, you’ll need to click the “Approve” button in the Duo app. Otherwise, you’ll still need to provide the account password.

    The Duo documentation has an excellent flow diagram for the authentication process using Duo, reproduced below.

    The steps are1:

    1. SSH connection initiated.
    2. Primary authentication: username/password or private key.
    3. Duo Unix connection established to Duo Security over TCP port 443.
    4. Secondary authentication via Duo Security’s service.
    5. Duo Unix receives authentication response.
    6. SSH session logged in.

    Comparison of Duo and TOTP

    So how does Duo compare to the Time-based one-time password (TOTP) solution we configured before?

    • TOTP uses a secret to generate time-based passwords. The “server” and the “client” run the same algorithm and generate the same password based on the common secret and the time.
    • The “server” and the “client” processes run independently and do not require internet.
    • Duo PAM module sends an authentication request to a Duo server and, therefore, requires an internet connection. You have a choice to make where allow or deny authentication if the Duo server is unavailable. You also will have to allow each server that uses Duo to have outbound connection to the internet.
    • Because Duo is a centrally managed platform, unlike TOTP, auditing login attempts. Unless you set up a syslog server, there’s no central way to manage TOTP login attempts.
    • Unless you’re using a centralized authentication server, such as RADIUS or AD, there’s no easy way to lock a user out when using TOTP. You’d have to touch every server where the user is configured. With Duo, you can disable the user and they’ll fail the second authentication factor (Duo) and you’ll be able to monitor those failures.

    Overall, I think Duo is an excellent solution for centralized MFA. I prefer it for the ease of set up, large number of integrations, and simplicity of centralized audit and management. But for the servers with no internet access, I’d fall back to TOTP, if I need the second factor.

    N.B.

    Our set up enables 2FA for remote SSH login ONLY. It does not provide any additional factors of authentication for the local login. Duo PAM can be configured to provide 2FA for local access and even require a push notification approval for sudo.

    So why did I choose not to protect the local logins with the second factor? Security always needs to be weighted against convenience of use. In my use case, there’s no expectation of physical shared access to the servers. And if an adversary has physical access to the machine, they can always reboot it, get into the recovery menu, change the root password, and do other nasty things. Unless we put additional measures to stop that, which will put additional barriers to the normal use. Again, we’re balancing security with convenience. In the case where there’s no physical access to the machines, I think securing just the remote SSH access with 2FA is perfectly reasonable. Your mileage may vary…

    1. https://duo.com/docs/duounix ↩︎
  • SSH.

    Securing Linux SSH with cryptographic keys.

    Since our Debian server is headless, we probably access it through SSH, which stands for Secure SHell. Now it’s time to make it even more secure.

    The best practice says you should set up SSH keys and disable the password login. This way the authentication is done with public-private cryptography. Which is pretty secure. And disabling password authentication makes it immune to the bruteforce login attempts.

    To generate the private-public key pair, on your Linux (or a Mac) computer, open the terminal and type ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/<SERVER_HOSTNAME>, where <SERVER_HOSTNAME> is the server name. You can name it whatever you want.

    Now, we copy the key to our server with the following command, where <SERVER_USERNAME> is the username of the user you’d be logging in to your server with, and <SERVER_IP> is your server’s IP address:

    ssh-copy-id -i ~/.ssh/<SERVER_HOSTNAME>.pub <SERVER_USERNAME>@<SERVER_IP>

    Now to access your server with the keys, we use: ssh -i ~/.ssh/<SERVER_HOSTNAME> <SERVER_USERNAME>@<SERVER_IP> .

    Next step is to disable password authentication for SSH. To accomplish this, we edit the /etc/ssh/sshd_config file with a command sudo nano /etc/ssh/sshd_config. Now make sure the file has the following lines:

    PasswordAuthentication no
    UsePAM no
    PermitRootLogin no

    Or with a single command:

    sudo sed -i'.bak' -e '/^PermitRootLogin/c PermitRootLogin no' -e '/^UsePAM/c UsePAM no' -e '/^#PasswordAuthentication/c PasswordAuthentication no' /etc/ssh/ssd_config

    You can change the port SSH listens on to something different. To do so, uncomment this line #Port 22 (by removing the # symbol) and change the number to something else. I recommend using a port number in the range 49152–65535. This way, somebody trying port 22 won’t be able to access your server through SSH because SSH is listening on another port. This is what’s called security by obscurity. It is not a best practice as it doesn’t necessarily make your system more secure but may stop script kiddies from trying. But they should also be stopped by all the other actions we’re taking.

    And now reload SSH with sudo systemctl reload ssh.

    ufw – uncomplicated firewall

    Now secure the server with a firewall. You can use iptables, but I prefer ufw, as it’s pretty uncomplicated, and it’s just a front end for iptables anyways. To install ufw we run

    sudo apt update && sudo apt install ufw -y

    Now allow SSH through the ufw firewall (where AAA.BBB.CCC.DDD is the ip you want to allow through ufw, or put any, if you want to open the port to any ip. You can use ssh in port if you kept it default, or change it to the value you change the SSH server listening port to). Make sure you do this step before enabling ufw as doing otherwise will lock you out and will have to figure out a way to console into your server. ufw blocks all incoming connections by default and without punching this hole first you won’t have any access from the network:

    sudo ufw allow from AAA.BBB.CCC.DDD to any port ssh proto tcp
    sudo ufw enable

    We could stop here. But why? Let’s install fail2ban and ban those IPs trying to login to our secure server without our authorization.

    fail2ban

    Install fail2ban and make sure it’s enabled on start with:

    sudo apt update && sudo apt install fail2ban -y
    sudo systemctl enable --now fail2ban

    After it’s done, let’s configure it. First, backup the config file

    sudo cp /etc/fail2ban/jail.jail /etc/fail2ban/jail.local

    And now edit it with sudo nano /etc/fail2ban/jail.local.

    When you get to the line ignoreip = make sure you include your IP so you won’t be locked out.

    Fort the ban times adjust the default parameters how you see fit:

    # "bantime" is the number of seconds that a host is banned.
    bantime  = 1d
    
    # A host is banned if it has generated "maxretry" during the last "findtime"
    # seconds.
    findtime  = 30m
    
    # "maxretry" is the number of failures before a host get banned.
    maxretry = 3

    Next makes sure backend says backend = systemd. This is required for the systems with systemd, such as Debian (since Debian 9 “stretch”).

    Next scroll down to the # JAILS section and make sure your [sshd] is as follows:

    [sshd]
    enabled = true
    port    = ssh
    logpath = %(sshd_log)s
    backend = %(sshd_backend)s

    To permanently ban repeat offenders we edit the [recidive] section as such:

    [recidive]
    enable=true
    logpath  = /var/log/fail2ban.log
    banaction = %(banaction_allports)s
    bantime  = -1
    findtime = 1d
    maxretry = 4

    It permanently bans (bantime = -1) the IPs if they try and fail to access our server 4 times (maxretry = 4) within time period of 1 day (findtime = 1d).

    To check the status of fail2ban and sshd jail use the commands:

    sudo fail2ban-client status
    sudo fail2ban-client status sshd

    We could stop here. But why? Let’s enable 2-factor authentication with TOTP.1

    Multi-factor Authentication with time-based one-time password

    First, install the necessary packages.

    sudo apt update && sudo apt install libpam-oath oathtool qrencode -y

    Next, generate a long random HEX number:

    dd if=/dev/random bs=1M count=1 status=none | sha256sum | cut -b 1-30

    Now, edit the file /etc/users.oath with the user (<SERVER_USERNAME>) who should have 2FA enabled, use the number generated with the command above (18c34c49adb7fdacf67d53d0bb7339 in our example, make sure you use your output). You can have multiple users in this file, but each user needs to have its own line.

    sudo nano /etc/users.oath
    HOTP/T30        <SERVER_USERNAME>  -       18c34c49adb7fdacf67d53d0bb7339

    Make sure to put the actual username in place of <SERVER_USERNAME>.

    Let’s generate the QR code.

    qrencode --type=ANSIUTF8 otpauth://totp/<SERVER_USERNAME>?secret=$( oathtool --verbose --totp 18c34c49adb7fdacf67d53d0bb7339 --digits=6 -w 1 | grep Base32 | cut -d ' ' -f 3 )\&digits=6\&issuer=<SERVER_HOSTNAME>\&period=30

    A few comments and explanation of the command parameters:

    <SERVER_USERNAME> – username of the user you’re enabling 2FA for

    18c34c49adb7fdacf67d53d0bb7339 – string from above

    --digits=6 – number of digits for the 2FA code. You can use 6,7, or 8, but even after about an hour of trying, I couldn’t make it work with anything other than 6. 6 is perfectly adequate.

    issuer=<SERVER_HOSTNAME> – not required, but it’s useful to see in the Authenticator app.

    Scan the QR code with your authenticator app on your phone.

    Now edit /etc/ssh/sshd_config by running sudo nano /etc/ssh/sshd_config and change the lines to:

    KbdInteractiveAuthentication yes
    UsePAM yes
    AuthenticationMethods publickey,keyboard-interactive
    #ChallengeResponseAuthentication yes

    Next, edit /etc/pam.d/sshd by running sudo nano /etc/pam.d/sshd. Comment out @include common-auth by putting a # symbol in front of it (otherwise SSH would ask you for the password in addition to the SSH keys and the second factor time code), and include auth requisite pam_oath.so usersfile=/etc/users.oath window=5 digits=6 below @include common-account.

    Now restart sshd and you should be all set:

    sudo systemctl restart sshd

    It was a lot of text and a lot of steps. We installed a few applications and configured our server to use SSH keys, disabled password authentication, installed and enabled firewall, installed fail2ban to ban IPs trying to access our server, and added another layer of security by utilizing 2-factor authentication. It was a lot of work. There’s no simple wizard that would do it for us. But hey. It’s free, at least…

    1. I used this website for 2FA setup. But I needed to adjust it to make it work on Debian 12 https://www.technomancer.com/archives/503. ↩︎