In VO, the following _dll declaration is correct:
_DLL function zLibCompress( ptrDest as ptr, nBytesOut ref long, ptrSource as ptr, nBytesIn as long ) as int:zlib.compress
This function has the C calling convention (comes from the Linux/Unix world), and therefore the pascal calling convention is not specified.
In X# this declaration does not compile, you have to add the strict calling convention
_DLL function zLibCompress( ptrDest as ptr, nBytesOut ref long, ptrSource as ptr, nBytesIn as long ) as int strict:zlib.compress