libnix is used in conjuction with m68k-amigaos-toolchain project. It's standard C link library (similar to glibc or newlib, but quite incomplete) for gcc cross compiler targeting AmigaOS/m68k systems.
There are several initiatives trying to bring up an updated C link library for AmigaOS/m68k systems: