The following are some tips on getting FreeBSD NIS servers to work with GlibC (GNU/Linux) clients. Getting them working took me several hours, and a lot of Google searching. If I've forgotten something, or have a pointer I should include, drop me a line.
The systems I was working with were:
First of all, I should state that the ypservenvironment I was dealing with was non-standard. The NIS maps (e.g., those in /var/yp/ domainname) were written out by custom tools (passwords were pulled from a database).
FreeBSD for itself uses master.passwd.{byname,byuid} maps. GlibC, however, needs shadow maps. Additionally, the layout for lines of master.passwd(5)are different from shadow(5). You need to make sure that when the master.passwd.{byname,byuid} maps are written out, that you also have shadow.{byname,byuid} maps are also written out.
This one was a real bugger. When writing out the passwd.{byname,byuid} maps, use a xinstead of FreeBSD's *(asterisk) for the 'nulled' password field. Seems quite dumb of GlibC to fail because of this, but it does.
Modern FreeBSD and GlibC systems can both use MD5 for their hashed passwords. I eventually ended up being able to use MD5 with no additional problem over DES.
I'll assume that you want FreeBSD-like NIS behavior; that is, you want some control over who actually allowed to login to the system (e.g., a subset of your NIS users).
I stronglyrecommend keeping an unused root shell open the whole time while working on the client.
In this file, have lines with:
passwd: files compat
shadow: files nis
group: files nis
You don't want nisfor the passwd entry, because that allows anyone in the NIS passwd map to login. The nisentries effectively put a +::::::entry at the bottom of files they relate to. That's fine for shadow and group, but not passwd; with a compatentry you get the better FreeBSD-like behavior of being able to put specific-user entries in your /etc/passwd, such as:
+ftobin::::::/bin/zsh
+cwenham::::::
Don't use authconfig(8)! It will change your /etc/nsswitch.conf compatentries back to nis, without your realizing it!
This was the easiest thing to do. Edit /etc/yp.conf appropriately. There's a manpage ( yp.conf(5)) to help. Keep in mind that the client is different from the FreeBSD ypbindclient.
You can start and keep running ypbind(8)during your entire configuration process. You shouldn't need to restart it or anything.
If I have remembered everything correctly, you should be all set to go. Make sure you look at the log files on both the client and server to help see what you might have missed.
There are two things that seemed to allow this to happen. First, in /etc/nsswitch.conf, make sure that you have compatfor your passwd, as described earlierin this document.
Second, if you are coming in via ssh, you might need to 'reload' sshd. I'm not sure why it does this, but sshd seems to cachepasswd entries, sometimes allowing me to login even after I'd removed my line from /etc/passwd.