The sender credentials can be encrypted with the rest of the message, leaving only the recipient address readable by the Signal server. Even if Signal wanted they couldn't see who is talking with who by this mean.
Signal is free and open source software, enabling anyone to verify its security by auditing the code. It's the only private messenger that uses open source, peer-reviewed cryptographic protocols to keep your messages safe.