If you can't make it work, you can use the diagnostic tool (there is a button on top of the plugin settings page), and take a close look on the «Prosody check» section.
Users that are not connected to yourPeertubeinstance are joining the chat using "anonymous accounts" (they can freely choose a nickname, and will be assigned a random avatar).
You can enable some external authentication methods to allowuser to create chat accounts.In such case their nickname and avatar will be automatically initialized with the remoteaccount information.
This also allows user to join the chat without creating Peertubeaccount (in case yourinstance has closed registration for example, or without waiting for account approval).