Fix SSH 'Permission Denied (publickey)' Errors: Ultimate Guide

Nov 16, 2025 by Admin 63 views

Hey guys, ever been stuck trying to connect to a remote server, probably a cool Google Cluster Container like you're setting up, only to be hit with that infuriating message: "Permission denied (publickey)"? Ugh, it's like the digital equivalent of a bouncer saying, "You're not on the list!" Trust me, you're not alone. This error is one of the most common head-scratchers for anyone diving into SSH, especially when dealing with multi-node setups like a master and worker architecture.

But don't sweat it! Today, we're going to demystify this error, break down why it happens, and give you a comprehensive, friendly guide to troubleshoot and fix it, especially in the context of your Google Cluster container environment. We'll cover everything from generating keys to understanding arcane file permissions, making sure your master can chat happily with its worker nodes. So, grab a coffee, and let's get those SSH connections flowing smoothly!

What's Up with "Permission Denied (publickey)"?

Alright, let's kick things off by really understanding what Permission denied (publickey) actually means. When you see this error, it's basically the SSH server on the remote machine telling your SSH client, "Hey, you tried to authenticate using a public key, but I'm not letting you in." It's not necessarily saying your key is invalid or corrupted; more often than not, it means the server either can't find your public key in its list of authorized keys, or it found it but something else is preventing the authentication from succeeding. Think of it like trying to use a key card to enter a building. If it says "Access Denied," it could mean your card isn't registered, your card is registered but expired, or maybe the card reader itself isn't working right. With SSH, it's usually a misconfiguration in how your public key is stored or how the server is allowed to read it.

Specifically, SSH public-key authentication is a super secure and convenient way to log into Linux servers without needing to type a password every single time. Instead of passwords, you use a pair of cryptographic keys: a private key (which you keep secret on your local machine) and a public key (which you share with the servers you want to connect to). When you try to connect, your client sends a request to the server, saying, "Hey, I'm user X, and here's my public key's fingerprint." The server then checks its authorized_keys file (usually located in ~/.ssh/authorized_keys for your user) to see if that public key is listed. If it is, the server challenges your client, which then uses your private key to cryptographically prove its identity. If everything matches up, boom, you're in! If not, you get our dreaded "Permission denied (publickey)" message.

Now, when you're working with a Google Cluster container, especially with a master node and several worker nodes, this scenario becomes even more critical. You need your master node to be able to securely and automatically connect to your worker nodes for orchestration, deployment, and data transfer. Manually entering passwords for each connection just isn't scalable or secure. So, ensuring your SSH public-key setup is flawless across all nodes is paramount. The error means one of these pieces of the puzzle isn't fitting: maybe the public key isn't on the worker node, perhaps its file permissions are too lax or too strict, or the SSH server itself isn't configured to allow public-key authentication for that user. Understanding this fundamental process is your first step towards becoming an SSH wizard and banishing this annoying error for good. So, let's dive into the common culprits and how to fix them!

The Core Culprits: Keys, Permissions, and Configuration

Alright, let's get down to the nitty-gritty. Most of the time, Permission denied (publickey) errors boil down to one of three things: issues with your SSH keys themselves, incorrect file permissions, or a server misconfiguration. We're going to tackle each of these head-on, giving you all the info you need to troubleshoot like a pro, especially in your Google Cluster setup.

Generating SSH Keys: Doing It Right from the Start

First things first, you need to make sure your SSH keys are generated correctly. This is the foundation of your secure connections. The ssh-keygen command is your best friend here, and using it correctly can save you a ton of headaches later. For your Google Cluster, you'll typically generate a key pair on your master node (or whatever machine you're initiating connections from) and then copy the public key to your worker nodes. Let's walk through it.

To generate a key pair, you'll use the command ssh-keygen. When prompted, it's often a good idea to accept the default file location (~/.ssh/id_rsa for RSA, ~/.ssh/id_ed25519 for Ed25519, etc.). This ensures consistency and makes it easier for your SSH client to find them. The prompt for a passphrase is super important. A passphrase adds an extra layer of security, encrypting your private key. If someone ever got hold of your private key, they still couldn't use it without the passphrase. For interactive logins, using a passphrase is highly recommended. However, for automated scripts or cluster orchestration where unattended access is needed (like a master node connecting to workers without manual intervention), you might consider generating a key without a passphrase. Just be aware of the security implications and ensure your private key file (id_rsa or similar) is extremely well-protected and never leaves the master node.

When choosing a key type, ssh-keygen defaults to RSA, but Ed25519 is generally recommended nowadays for its security and efficiency. So, a command like ssh-keygen -t ed25519 -C "your_email@example.com" is a solid choice. The -C flag allows you to add a comment, which is handy for identifying keys later, especially when you have multiple keys for different purposes. After running this, you'll end up with two files in your ~/.ssh/ directory: id_ed25519 (your private key) and id_ed25519.pub (your public key). Remember, your private key must remain secret and never be shared! The public key, however, is meant to be shared with any server you want to access.

What if you already have keys? Be very careful about generating new ones. If you run ssh-keygen and accept the default filename, it will ask if you want to overwrite existing keys. If you say yes, your old keys are gone, and you'll lose access to any servers configured with them. If you need a new key for a specific purpose (like your Google Cluster), consider using a different filename, for example: ssh-keygen -t ed25519 -f ~/.ssh/id_cluster_gcp -C "gcp_cluster_key". This creates id_cluster_gcp and id_cluster_gcp.pub, allowing you to manage multiple identities. This is a crucial tip for avoiding unexpected lockouts! So, making sure you have the right keys, generated securely, is your first big step towards success. Next, we'll talk about getting that public key onto your worker nodes correctly.

Distributing Your Public Key: The `authorized_keys` Magic

Okay, you've got your shiny new key pair generated on your master node. Now, the magic happens when you get your public key onto your worker nodes. This is where the remote server learns to trust you. The goal is to place your public key's content into a file named authorized_keys within the ~/.ssh/ directory of the user you want to log in as on the remote (worker) machine. There are a couple of ways to do this, but one stands out as the easiest and most reliable: ssh-copy-id.

Using ssh-copy-id (The Easiest Way):

Seriously, guys, if you have initial password-based access to your worker nodes, ssh-copy-id is your best friend. It handles all the fiddly bits of creating the .ssh directory, setting correct permissions, and appending your public key to authorized_keys.

From your master node, simply run:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@worker_node_ip

(Replace ~/.ssh/id_ed25519.pub with the path to your public key file, user with the username on the worker node, and worker_node_ip with the worker's IP address or hostname). It will prompt you for the user's password on worker_node_ip. Once you enter it correctly, ssh-copy-id does its job. If you used a custom key filename like id_cluster_gcp.pub, just make sure to specify it with the -i flag. After this, you should be able to SSH into that worker node without a password (or with your passphrase if you set one for your private key).

Manual Copying (When ssh-copy-id Isn't an Option):

Sometimes, ssh-copy-id isn't available, or you might need to do it manually. This method is a bit more prone to permission errors if you're not careful, but it's totally doable. First, you need to get the content of your public key. On your master node, run:

cat ~/.ssh/id_ed25519.pub

Copy the entire output. Then, you need to SSH into your worker node (perhaps using a password for the first time, or another method) and perform these steps:

Create the .ssh directory (if it doesn't exist):
```
mkdir -p ~/.ssh
```
Append your public key to authorized_keys:
```
echo "[PASTE YOUR PUBLIC KEY HERE]" >> ~/.ssh/authorized_keys
```
Alternatively, a more robust one-liner from your master node, assuming password access:
```
cat ~/.ssh/id_ed25519.pub | ssh user@worker_node_ip 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'
```
This command is super powerful as it creates the directory, sets its permissions, appends the key, and then sets the file's permissions – all in one go! Be very careful not to overwrite authorized_keys with just > if there are other keys already there. Always use >> to append. Once the key is in place, you still need to make sure the permissions are rock solid. Which brings us to our next critical point!

File Permissions: The Silent Killers of SSH Connections

Okay, guys, if you're pulling your hair out over "Permission denied (publickey)" and you've double-checked that your public key is indeed on the remote server in ~/.ssh/authorized_keys, then chances are file permissions are the culprit. SSH is extremely picky about permissions. It's a security feature: if the permissions are too lax, SSH assumes someone else could tamper with your keys, and it will simply refuse to authenticate you. This is, hands down, the most common reason people run into this error.

Let's break down the critical permissions you need to set on the remote (worker) node for the user you're trying to log in as:

Your Home Directory (~ or /home/user): The home directory itself shouldn't be group-writable or world-writable. Typically, this is set correctly by default, but it's worth checking. The permissions should be drwxr-xr-x or drwx------ (755 or 700). You can check with ls -ld ~ and fix with chmod 755 ~ if needed. SSH will sometimes complain if the home directory is too open.
The .ssh Directory (~/.ssh/): This directory must be private to the user. No one else should be able to write to it, and ideally, no one else should even be able to read its contents. The correct permissions are drwx------ (700). This means the owner can read, write, and execute (which means traversing into the directory), but nobody else (group or others) has any permissions. To set this, run:
```
chmod 700 ~/.ssh
```
The authorized_keys File (~/.ssh/authorized_keys): This file, which contains your public keys, must also be private to the user. It should not be writable by anyone other than the owner, and ideally, no one else should be able to read it either. The correct permissions are -rw------- (600). This means the owner can read and write, but nobody else has any permissions. To set this, run:
```
chmod 600 ~/.ssh/authorized_keys
```
Ownership: Equally important, the .ssh directory and the authorized_keys file must be owned by the user trying to log in. If root or another user owns these files, SSH will get suspicious and deny access. You can check ownership with ls -l ~/.ssh. If it's incorrect, fix it with:
```
sudo chown user:user ~/.ssh
sudo chown user:user ~/.ssh/authorized_keys
```
(Replace user with the actual username, e.g., chown yourusername:yourusername.)

Why is SSH so strict about this? Imagine if a malicious user could write to your authorized_keys file. They could add their own public key and gain access to your server! Or, if your private key on the client side (~/.ssh/id_rsa or similar) has too-loose permissions, someone could copy it. SSH's strict StrictModes (which is usually enabled by default in sshd_config) ensures that only the intended user has control over these critical authentication files. Neglecting these file permissions is a recipe for frustration, but once you nail them down, your SSH connections will be far more reliable and secure. So, always remember: 700 for the directory, 600 for the files, and correct ownership!

Troubleshooting Like a Pro: Digging Deeper into Your Google Cluster

If you've checked your keys, ensured proper distribution, and meticulously set those file permissions, but you're still getting "Permission denied (publickey)", then it's time to put on your detective hat. We need to gather more information. This section will guide you through deeper troubleshooting steps, with a special eye on your Google Cluster container environment.

Verbose SSH Client Output: Your Best Friend (`ssh -v`)

When things go wrong with SSH, your very best friend is the verbose output from your SSH client. This output provides a blow-by-blow account of the authentication process, often giving you critical clues about exactly where things are failing. Instead of just ssh user@remote, add a -v (for verbose) or even -vv or -vvv (for even more verbosity): ssh -vvv user@worker_node_ip.

What to look for in the output, guys? You'll see lines about key exchange, trying different authentication methods (password, publickey), and messages about whether a key was accepted or rejected. Here are some common snippets and what they mean:

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password: This tells you what authentication methods the server is willing to accept. If publickey isn't listed, that's a huge red flag – the server isn't even configured for public key auth!
debug1: Offering public key: /home/youruser/.ssh/id_rsa RSA SHA256:...: This shows your client offering a specific private key for authentication. Make sure it's the right one!
debug3: send_pubkey_test: no mutual signature algorithm: This is rare but means your client and server can't agree on a cryptographic algorithm for signing. Might indicate an old server or very strict client config.
debug1: Server accepts key: /home/youruser/.ssh/id_rsa RSA SHA256:...: This is a good sign! It means the server found your public key in authorized_keys and is willing to proceed with the challenge. If you still get "Permission denied" after this, it often points to a problem with your private key's passphrase or the key itself.
debug1: Authentications that can continue: publickey: If you see this followed by debug1: Next authentication method: publickey and then debug1: Trying private key: /home/youruser/.ssh/id_rsa, but then it cycles back to publickey or eventually gives up, it suggests the server isn't finding your public key in authorized_keys or the permissions on ~/.ssh or authorized_keys are wrong. The server essentially says, "I saw your key offer, but I don't recognize it as valid for this user, so try again."

Paying close attention to these verbose logs is like having a direct line to the SSH negotiation process. It will tell you if your client is even trying the right key, if the server is listening for public key authentication, and if the keys are being matched correctly. It's your first stop for getting concrete diagnostic information beyond just the generic error message.

Server-Side Checks: `sshd_config` and Logs

While the client-side verbose output is super helpful, sometimes the problem lies squarely on the server. For your Google Cluster worker nodes, you need to be able to check their SSH server configuration and logs. This is where you verify that the server is set up to allow public key authentication in the first place.

First, access your worker node (perhaps through the Google Cloud Console's built-in SSH feature, or if you still have password access). Once on the worker, you'll want to inspect the sshd_config file. This file usually lives at /etc/ssh/sshd_config. Open it with your favorite text editor (e.g., sudo nano /etc/ssh/sshd_config or sudo vi /etc/ssh/sshd_config).

Look for these critical lines and ensure they are uncommented and set correctly:

PubkeyAuthentication yes: This is non-negotiable. If this is set to no or commented out, public key authentication simply won't work. Make sure it's yes.
AuthorizedKeysFile .ssh/authorized_keys: This specifies where SSHD looks for the public keys. The default is usually ~/.ssh/authorized_keys, and .ssh/authorized_keys in the user's home directory is implied. It's good practice to keep the default unless you have a very specific reason not to. Sometimes you might see %h/.ssh/authorized_keys where %h is a placeholder for the user's home directory.
PermitRootLogin prohibit-password (or no): If you're trying to SSH as root, ensure this is set to something sensible. prohibit-password allows root login with keys but not passwords. If it's no, then root can't log in via SSH at all, even with keys. For cluster nodes, it's generally best practice to create non-root users and use sudo.
StrictModes yes: This is the setting that enforces those strict file permissions we talked about earlier. By default, it's yes, and you generally want to keep it that way for security. If you were desperate, temporarily setting it to no might allow you to connect with incorrect permissions, but it's a huge security risk and should never be done in production. Use it only for very temporary debugging and revert immediately.

After making any changes to sshd_config, you must restart the SSH service for them to take effect. On most Linux systems, you'd do this with sudo systemctl restart sshd or sudo service sshd restart.

Next, check the server's authentication logs. These logs often provide the server's perspective on why an authentication attempt failed. Common locations include:

journalctl -u sshd (for systems using systemd, like most modern Linux distros)
/var/log/auth.log (on Debian/Ubuntu-based systems)
/var/log/secure (on RHEL/CentOS/Fedora-based systems)

Look for entries around the time you attempted to connect. You might see messages like Authentication refused: bad ownership or modes for directory /home/user/.ssh or Authentication refused: bad ownership or modes for file /home/user/.ssh/authorized_keys, which directly point to permission issues. Or you might see User user not allowed because listed in DenyUsers if there are explicit access restrictions. These logs are incredibly valuable for narrowing down the problem, offering concrete error messages straight from the horse's mouth.

Google Cluster Specifics: Network, Firewalls, and Ephemeral Storage

Now, let's zoom in on the specific challenges you might face in a Google Cluster Container environment, like Google Kubernetes Engine (GKE) or similar setups. These environments introduce a few extra layers of complexity compared to a standalone VM.

First, Network Connectivity and Firewalls. Even if your keys and permissions are perfect, if your master node can't reach your worker nodes on the SSH port (default 22), you're going nowhere. In Google Cloud, this usually means checking your Firewall rules. Each GKE cluster often has default firewall rules, but if you've deployed custom networks or modified rules, you need to ensure traffic on TCP port 22 is allowed from your master node's IP range to your worker nodes' IP ranges. You can inspect your firewall rules using the gcloud compute firewall-rules list command or through the Google Cloud Console. Look for a rule that permits ingress (incoming) traffic to your worker node instances on port 22, originating from your master's network. Also, verify that your worker nodes actually have public IP addresses if you're connecting from outside the cluster's internal network, or that they are on the same VPC network for internal IP-based connections. Sometimes, a quick ping from the master to the worker IP can confirm basic network reachability, but firewalls often block ping (ICMP) even if SSH (TCP) is allowed.

Second, consider Ephemeral Storage and Container Lifecycles. This is a big one for containerized environments. When you generate SSH keys and place authorized_keys inside a container (like your master or worker nodes in a Google Cluster), where exactly are these files stored? If they're in ephemeral storage (meaning storage that's tied to the container's lifecycle and disappears when the container restarts or gets rescheduled), then every time your worker node container restarts, your .ssh directory and authorized_keys file might be wiped clean! This would instantly break your SSH connections and lead to the "Permission denied" error again and again.

To combat this, you'll need to use persistent storage or a robust provisioning strategy. Here are a few ways to handle it:

Persistent Disks/Volumes: For GKE, you could mount a PersistentVolumeClaim (PVC) to your worker nodes' containers, and store the .ssh directory there. This ensures the data persists even if the container restarts.
ConfigMaps or Secrets: For static public keys, you could store the content of authorized_keys in a Kubernetes ConfigMap or Secret and then mount that as a file into the ~/.ssh/ directory of your worker node containers. This is particularly useful for provisioning the same authorized_keys file across multiple worker nodes.
Init Containers/Entrypoint Scripts: You could use an init container or an entrypoint script within your worker node's Docker image to fetch the authorized_keys content from a secure location (like Google Secret Manager, a private Git repository, or an internal file server) and set up the .ssh directory and permissions at container startup. This makes your containers stateless and self-configuring.

Always double-check the IP address you're trying to connect to. In dynamic cluster environments, IP addresses can change, especially if nodes are scaled up/down or restarted. Ensure your master node (or whatever machine is initiating the connection) has the correct, current IP addresses for your worker nodes. DNS resolution within the cluster or using internal cluster service names can help abstract away changing IPs.

By systematically checking network connectivity, understanding the ephemeral nature of containers, and implementing persistent storage or smart provisioning for your SSH configurations, you can head off a lot of the "Permission denied" issues specific to cluster environments. It's all about making sure that ~/.ssh/authorized_keys is not only present but persists and has the correct permissions throughout your cluster's lifecycle.

A Step-by-Step Walkthrough for Your Master-Worker Setup

Alright, guys, let's put all this knowledge into action with a practical, step-by-step guide tailored for your Google Cluster Container setup with one master node and two worker nodes. We're going to assume you have some initial access to all these nodes (either via password, Google Cloud Console SSH, or another temporary method) to perform the initial setup. This walkthrough will ensure your master node can seamlessly connect to your worker nodes using SSH keys.

Phase 1: On Your Master Node (The Initiator)

Generate Your SSH Key Pair: First, log into your master node. We'll generate a secure Ed25519 key pair. It's a good idea to use a descriptive comment.
```
ssh-keygen -t ed25519 -f ~/.ssh/id_cluster_gcp -C "gcp_master_key"
```
When prompted, you can choose to enter a passphrase for extra security. If this key is for automated processes that can't handle a passphrase, you can leave it empty, but be extra vigilant about protecting the private key. This will create two files: ~/.ssh/id_cluster_gcp (your private key) and ~/.ssh/id_cluster_gcp.pub (your public key).
Verify Private Key Permissions: Ensure your private key (id_cluster_gcp) has strict permissions. It should only be readable by you.
```
chmod 600 ~/.ssh/id_cluster_gcp
```
Also, ensure your ~/.ssh directory has the correct permissions:
```
chmod 700 ~/.ssh
```
Start SSH Agent (Optional but Recommended): If you're using a passphrase, ssh-agent will save you from typing it repeatedly. Add your key to the agent:
```
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_cluster_gcp
```
You'll be prompted for your passphrase if you set one.

Phase 2: Distribute Public Key to Worker Nodes (The Targets)

Now, for each of your worker nodes (let's say worker1_ip and worker2_ip), you need to get the public key (id_cluster_gcp.pub) onto them. We'll use the robust one-liner that handles directories and permissions.

For Worker Node 1: From your master node, execute this command. Replace worker_user with the username you want to log in as on the worker (e.g., ubuntu, debian, gcpuser, etc.) and worker1_ip with its actual IP address or hostname. You'll be prompted for the worker_user's password on worker1_ip for this initial connection.
```
cat ~/.ssh/id_cluster_gcp.pub | ssh worker_user@worker1_ip 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'
```
This command creates the .ssh directory (if it doesn't exist), sets its permissions to 700, appends your public key to authorized_keys, and then sets authorized_keys permissions to 600. It's a complete package!

For Worker Node 2: Repeat the same process for your second worker node, replacing worker1_ip with worker2_ip.

cat ~/.ssh/id_cluster_gcp.pub | ssh worker_user@worker2_ip 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

Phase 3: Verify and Test Connections

After setting up both worker nodes, it's testing time!

From Master to Worker 1: Try to connect using your specific private key. If you're using ssh-agent, it should just work. If not, explicitly specify the key:
```
ssh -i ~/.ssh/id_cluster_gcp worker_user@worker1_ip
```
You should now connect without being prompted for a password (only your passphrase if you set one for the private key and haven't added it to ssh-agent). If you still get Permission denied, go back to the verbose logging (ssh -vvv) and server logs we discussed earlier.
From Master to Worker 2: Repeat the test for your second worker node.
```
ssh -i ~/.ssh/id_cluster_gcp worker_user@worker2_ip
```

Phase 4: Google Cluster Container Persistence Consideration

Remember our discussion about ephemeral storage? If your worker nodes are containers that might restart and lose their filesystem changes, you need to bake this authorized_keys setup into your deployment strategy.

For GKE: Consider using a ConfigMap or Secret in Kubernetes to hold the public key content. Then, mount this ConfigMap as a volume into ~/.ssh/authorized_keys in your worker node pods. Ensure the fsGroup or securityContext settings in your Pod spec apply the correct chmod and chown after mounting, as Kubernetes mounts might initially have incorrect permissions. Alternatively, use a PersistentVolumeClaim (PVC) for the .ssh directory, but this adds complexity and might not be ideal for homogeneous worker nodes. A ConfigMap approach is often simpler for authorized_keys.

By following these steps meticulously, paying close attention to file paths, usernames, IP addresses, and especially those critical file permissions, you should be able to get your SSH connections working flawlessly across your Google Cluster. It might seem like a lot of steps, but each one is crucial for both security and functionality. You've got this!

Wrapping Up: Don't Let SSH Frustrate You!

Phew! We've covered a lot of ground today, guys, tackling the notorious "Permission denied (publickey)" error head-on. From understanding the basics of public-key authentication to diving deep into file permissions, sshd_config settings, and even the unique challenges of a Google Cluster container environment, you're now armed with the knowledge to troubleshoot and fix this pesky issue like a true pro.

The biggest takeaways? Always, always check your file permissions! The ~/.ssh directory should be chmod 700, and the ~/.ssh/authorized_keys file should be chmod 600, both owned by the connecting user. Beyond that, leveraging verbose SSH client output (ssh -vvv) and inspecting server-side logs (like journalctl -u sshd) are your best diagnostic tools. And for those of you in dynamic containerized environments like Google Clusters, remember the importance of persistent storage or smart provisioning to ensure your SSH configurations survive container restarts.

SSH is a fundamental tool in the Linux and cloud world, and while it can be frustrating when it doesn't work, most errors have clear, logical solutions. By systematically going through the steps we've discussed – generating keys correctly, distributing them properly, nailing those permissions, and digging into logs when needed – you'll build robust, secure connections every time. So, go forth, connect confidently, and make your Google Cluster sing! You've successfully conquered the SSH permission dragon. Keep learning, keep experimenting, and happy SSH'ing!