If you’re setting up a Jenkins SSH connection on macOS and getting a “Can’t connect to server” error, the problem is almost certainly your SSH key format. Newer macOS versions generate keys in OpenSSH format by default, which Jenkins doesn’t recognize. The fix takes one command.
The Error
You generate an SSH key on your Mac, add the private key to Jenkins credentials, configure the SSH remote host, and hit Test Connection. Instead of “Successful”, you get:
Can't connect to serverNo detailed error, no hint about what went wrong. The SSH key looks fine, the remote host is reachable, and the public key is deployed — but Jenkins refuses to connect.
Before the Fix: Jenkins Connection Test Failing
Here’s what the error looks like in Jenkins when using the default OpenSSH key format:
The Cause
Since macOS Mojave (10.14), ssh-keygen generates private keys in OpenSSH format by default. You can tell by looking at the first line of the key file:
-----BEGIN OPENSSH PRIVATE KEY-----Jenkins (and many older SSH libraries built on JSch) can’t read this format. They expect the traditional PEM format, which starts with:
-----BEGIN RSA PRIVATE KEY-----The Fix: Generate an RSA Key in PEM Format
1. Generate the key
ssh-keygen -t rsa -m PEM -f ~/.ssh/jenkins-key-t rsa— creates an RSA key (widely compatible)-m PEM— forces PEM output format instead of OpenSSH-f ~/.ssh/jenkins-key— saves the key pair to a specific file
You’ll be prompted for a passphrase. For Jenkins automation, you can leave it empty (press Enter twice) or set one and store it in Jenkins credentials.
2. Set permissions
chmod 600 ~/.ssh/jenkins-key3. Deploy the public key to the remote host
Copy the public key to the server Jenkins needs to connect to:
ssh-copy-id -i ~/.ssh/jenkins-key.pub user@203.0.113.10Or manually append it to ~/.ssh/authorized_keys on the remote host. For SSH key setup on EC2 instances, see How to Setup Passwordless SSH Login on EC2 Ubuntu 22.04.
4. Add the key to Jenkins
- Go to Manage Jenkins > Credentials
- Add a new SSH Username with private key credential
- Paste the contents of
~/.ssh/jenkins-key(the private key, not the .pub file) - Set the username to match the remote host user
To view the private key contents for pasting:
cat ~/.ssh/jenkins-keyConfirm it starts with -----BEGIN RSA PRIVATE KEY----- (PEM format). If it says -----BEGIN OPENSSH PRIVATE KEY-----, Jenkins won’t accept it.
5. Test the connection
Go to your SSH remote host configuration in Jenkins and click Test Connection. It should now return Successful.
After the Fix: Jenkins Connection Test Passing
After switching to the PEM-format key, the same Test Connection button now works:
Convert an Existing Key to PEM Format
If you already have a key deployed to remote servers and don’t want to redeploy, you can convert the existing private key to PEM format in place:
ssh-keygen -p -m PEM -f ~/.ssh/jenkins-key-p— changes the passphrase (or format) of an existing key-m PEM— converts the output to PEM format
This converts the private key file without changing the public key, so your authorized_keys entries on remote hosts stay valid.
Quick Diagnostic Checklist
If Test Connection still fails after the key format fix, check these:
| Check | How to Verify |
|---|---|
| Key format | Private key starts with -----BEGIN RSA PRIVATE KEY----- |
| Public key deployed | Public key is in ~/.ssh/authorized_keys on the remote host |
| SSH port open | Port 22 is reachable from the Jenkins server (nc -zv host 22) |
| Username correct | Jenkins credential username matches the remote user |
| File permissions | Remote ~/.ssh is 700, authorized_keys is 600 |
Conclusion
The “Can’t connect to server” error in Jenkins on macOS is a key format problem, not a network or permission issue. Generating your SSH key with -m PEM produces a format Jenkins can read. If you have an existing key, convert it with ssh-keygen -p -m PEM without needing to redeploy.
For related SSH topics, see How to Configure Git Pushes Without Authentication Prompts which also covers SSH key setup for automation.
