As an example, I’ve been trying to send a “SID Skip Patch” command to my SID station. The SysEx message for this command is F0 00 20 3C 01 00 03 F7.

The following code shows how to construct this message in Java (example based on SendSysex.java at jsresources.org):


String s = "F000203C010003F7";
int n = s.length() / 2;
byte[] msg = new byte[n];
for (int i = 0; i < n; i++) {
    msg[i] = (byte) Integer.parseInt(
    s.substring(i * 2, i * 2 + 2), 16);
}


The next step is to send the resulting byte array to a MIDI output. The following code works fine (tested with MIDI monitor and on a real SID station through an ESI MIDI Mate):

String[] outputs = de.humatic.mmj.MidiSystem.getOutputs();
int index = 0;
 
// insert code to set correct index here...
 
de.humatic.mmj.MidiOutput mo =     de.humatic.mmj.MidiSystem.openMidiOutput(index);
mo.sendMidi(msg);

I’ve left out the code to determine the correct index as it is irrelevant. This could be a dropdown list in a GUI or some substring search or whatever. The following code tries to do the same thing using Snow Leopard libraries, but sends nothing at all:

javax.sound.midi.SysexMessage sysexMsg;
javax.sound.midi.Receiver receiver;
javax.sound.midi.MidiDevice.Info[] dev;
int index;
javax.sound.midi.MidiDevice myOutPort;
 
sysexMsg = new SysexMessage();
sysexMsg.setMessage(msg, msg.length);
dev = javax.sound.midi.MidiSystem.getMidiDeviceInfo();
 
// insert code to set correct index here...
 
myOutPort =
    javax.sound.midi.MidiSystem.getMidiDevice(dev[index]);
myOutPort.open();
receiver = myOutPort.getReceiver();
receiver.send(sysexMessage, -1);

If you have succeeded in making this work, please let me know. MMJ development seems to have stopped so I’d kind of want to start using OSXs own stuff…

Summarizing the steps:

1. Download and mount Wireshark for 10.5 Leopard Intel
2. Drag Wireshark to Applications
3. Open a terminal and type:
   defaults write com.apple.finder AppleShowAllFiles YES
   killall Finder
   - com.apple.Finder (i.e. Finder with a capital F) did not work for me!
   - For more info on defaults type man defaults.
4. Create /usr/local/bin/ if you don’t already have one.
   
5. Drag the Command Line utilities into /usr/local/bin/.
6. Drag ChmodBPF folder to Startup Items.
7. In the terminal type:
   sudo chown -R root:wheel ChmodBPF
8. Launch Wireshark
9. Add /usr/share/snmp/mibs/ under Edit → Preferences → Name Resolution → SMI (MIB and PIB) paths → Edit → New
10. Reboot, done!

In this article I’ll describe how to setup a home audio server on Ubuntu. I tried it on kernel 2.6.22-14, but it will probably work similarly on other systems. The client audioplayer in my setup is iTunes running on Mac OSX 10.4 (Tiger). There’s a number of articles available online, describing how to do this, but somehow none of them seemed to really cut the mustard for me. I’ll start with a short inventory:

- Running an iTunes server on Ubuntu
- Streaming iTunes from Ubuntu
- Using Ubuntu to share with Firefly (mt-daap)
- How to share your music via iTunes on the Net

After some investigation and hacking, I arrived at the following setup:

Step 1: Update and upgrade
Make sure you got all the relevant upgrades and updates for your Ubuntu machine, by typing at a command prompt:

sudo apt-get update
sudo apt-get upgrade

Step 2: Format your external harddisk
If you’re a Mac adept like me, you will want to format your kickass external harddisc to Apple’s HFS+ filesystem. To do so, attach it to a Mac, open Applications -> Utilities -> Disk Utility and Erase the Disk using Volume Format HFS (“Mac OS Extended”). I recommend not to use journaling. Although it helps recovering your data in case of a corrupt disk, I found that Ubuntu support for HFS+ Journaled disks is still a bit shaky.

Step 3: Move your music to central storage
Now go attach your external disk to all your computers, move or copy all your mp3s onto it, and attach the disk to your Ubuntu machine.

Step 4: Install a DAAP server
iTunes speaks and listens to a protocol called DAAP (Digital Audio Access Protocol) for sharing media across a network. By default, DAAP uses TCP port 3689. The DAAP server is the sofware that will spit out your audio onto the network. We will install Firefly on the Ubuntu machine for this purpose. Firefly, or mt-daap as it used to be called, requires some additional packages, install them by typing the following at a command prompt:

sudo apt-get install howl-utils libhowl-dev libhowl0
sudo apt-get install mt-daapd

Now you’re ready to start the server with:

sudo mt-daapd &

Step 5: Configure DAAP server
Open a webbrowser and goto http://your_ubuntu_machine:3689/
If you installed correctly, the server will prompt you with an authentication window. By default, don’t specify a username and use mt-daapd as password.

Go to the Configuration tab and change the default Admin password. It’s probably illegal to open up your music collection online for the whole world to see, so choose a Music password as well.
Set the Music Folder to a directory on the harddisk that you just stuffed with your music collection. I started out with a small test directory first, but if that works just scan your whole database. To verify that Firefly actually found all your files, go to a command prompt and check how many files you’ve really got:

find . -type f | wc -l

When I was setting things up, I found that I had to fiddle around with file permissions and ownership before the Firefly admin console would see them.

Step 6: Test on local network
Right now you should already be able to connect to your music server within your private network. Open up iTunes and check that a Shared Playlist is shown:

Step 7: Port forwarding on router
Go your router’s admin console and setup a virtual server for port 3689. All incoming traffic on port 3689 should be routed to your Ubuntu machine.

Step 8: Installing rendezvous proxy
iTunes cannot find your shared playlist from another network. You need to configure a proxy on your client machine that will mimic a shared playlist to iTunes, and connect to your home network on the other side. You can use the rendezvous proxy for this. Get it here or google for it Download and mount the .dmg file and run the application contained in it. You should end up with a new icon in the taskbar.

Click on it, select Preferences… and then Add Host.

Fill in your home IP address and port 3689, type a dummy host label and a descriptive Service Name, and select Service Type as shown.

Step 9:Test on remote network
You’re all set! Fire up iTunes on your laptop while lying on a sunny beach far away, using the WiFi network of the local cocktail bar, and listen to your favourite Slayer tracks while your Ubuntu server is streaming from the comfort of your home

The application jVoiceBridge is a software-only audio mixer that handles Voice over IP (VoIP) audio communication and mixing, for tasks such as conference calls, voice chat, speech detection, and audio for 3D virtual environments. Currently it is most commonly known for its use in the Wonderland project, a 3D virtual environment developed by Sun.

In this article I will give an introduction to the design of the jVoiceBridge software. I am not providing a user manual, nor will I be giving detailed operating instructions; if you’re looking for that information, have a look at the jVoiceBridge WiKi. Instead, I’ve been reverse engineering the source code and I will provide some diagrams and explanations which I drew up during my exploration. It won’t cover every detail, but it should be enough to a give you a general idea of how the code is structured.

Many thanks go out to Joe Provino from Sun for patiently answering all my questions!

Important terms and abbreviations

Overview

The following pictures give an overview of the most important classes and packages related to handling commands which are received over the telnet connection.

The RequestHandler is the main entry point into jVoiceBridge. As soon as the bridge accepts a new telnet connection, it creates a new RequestHandler thread for reading socket input data. The RequestHandler dispatches the data to the RequestParser, until a newline is received. At that point, the RequestHandler creates the appropriate CallHandler for handling the call.
A request is a command, followed by parameters and separated by colons. The RequestParser distinguishes different types of requests. Some requests are parsed and handled immediately (parseImmediateRequest), while others are parsed and executed upon reception of a newLine (parseCallParameters).
If the request matches none of the immediate commands, the parseImmediateRequest method returns false, and RequestParser will just try to parse call parameters from the supplied input. Call parameters are collected in a CallParticipant object.
If none of these call parameters match the given request, finally an attempt is made to parse the request as a twoparty-call request. If that fails, an exception is thrown (“Illegal request”). Communication towards the end user is handled by the RequestHandler‘s writeToSocket method.

The jVoiceBridge WiKi provides an extensive list of all immediate requests and all call parameters. Some requests are related to actual call behaviour – for example the createConference or mute requests. Others are bridge configuration commands, for example defaultSipProxy or rtpTimeout.
To start understanding the design, I recommend to focus on some important requests. The following requests are dispatched by the RequestParser to static ConferenceManager methods:

• createConference
• createWhisperGroup
• destroyWhisperGroup
• endConference
• removeConference
• recordConference
• playTreatmentToConference

The following requests are dispatched by the RequestParser to static CallHandler methods:

• mute
• muteWhisperGroup
• muteConference
• playTreatmentToCall

The following requests are dispatched by the RequestParser to a specific CallHandler instance:

• privateMix: delegated to the ConferenceMember of the CallHandler identified by the supplied call id.
• removeCallFromWhisperGroup: delegated to the ConferenceMember of the CallHandler of the supplied call id.
• addCallToWhisperGroup: delegated to the CallHandler for the supplied call id.
• whisper: delegated to the ConferenceMember of the CallHandler of the supplied call id.

The following requests are some examples of bridge configuration commands:

• rtpTimeout
• sendSipUriToProxy
• defaultSipProxy
• voIPGateways

An example: starting a new Conference

I recommend that you have the jVoiceBridge source code in front of you while reading this section. I will use a typical example to walk through the code: the creation of a new conference. We start by imagining that a user has started a telnet connection to the bridge, and entered the following request:

$telnet_prompt:> createConference = testid:PCM/44100/2

No conference with this id exists yet, so a new ConferenceManager is created and added to the ConferenceManager‘s static list of ConferenceManagers. Media settings and name are set and ConferenceSender and ConferenceReceiver are created. The conference id is stored in the CallParticipant. The following sequence diagram illustrates this behaviour.

In case of synchronous mode, all that is returned to the telnet client is “END — SUCCESS“. If synchronousMode is false, RequestHandler will only respond to the telnet client when one of its CallEventListener methods is called.

$telnet_prompt:> (newline)

The newline will cause the RequestHandler to do a validateAndAdjustParameters first. That will report to the user (among other things) that no phone number was specified! The user specifies one by typing:

$telnet_prompt> phoneNumber = sip:joe@host.com

…followed by:

telnet_prompt> (newline)

RequestHandler will determine what to do next, by looking at the CallParticipant. Its cp.remoteMediaInfo is still null, cp.migrateCall is false, secondPartyNumber was not set (only set for a 2-party call request or a call migration), so an OutgoingCallHandler is created and started.

The OutgoingCallHandler

The following picture shows some important classes which are involved during creation of a conference.

OutgoingCallHandler asks the ConferenceManager for this CallParticipant‘s conferenceManager – this had just been created. OutgoingCallHandler calls conferenceManager.joinConference(cp), upon which the conferenceManager sets some audio treatments in the CallParticipant and creates a new ConferenceMember object for this CallParticipant. The ConferenceMember is then:

• added to the conferenceManager‘s list of conference members,
• added to conferenceManager’s ConferenceReceiver, so it will be registered to a java.nio.DatagramChannel. Notice that the ConferenceReceiver thread starts as soon as it’s created.
• returned to the OutgoingCallHandler.

The OutgoingCallHandler maintains a static Vector of active calls. It adds itself to this Vector, so the right CallHandler can be found later when a call is hung up.
The described sequence of events is illustrated in the following picture:

The OutgoingCallHandler then examines if we’d specified any gateways (not in this example) and calls placeCall(). The following sequence diagram shows what happens then.

The Bridge‘s default protocol is used, so a SipTPCCallAgent (Third Party Call) is created. Upon construction, the SipTPCCallAgent retrieves and remembers the MediaInfo from the OutgoingCallHandler‘s conferenceManager. It creates a SipUtil, which in turn constructs a SdpManager based on the MediaInfo passed in (see below for more info).

Finally, the OutgoingCallHandler calls SipTPCCallAgent.initiateCall();

SipTPCCAllAgent changes to state CallState.INVITED, and asks the OutgoingCallHandler for its memberReceiver‘s InetSocketAddress. This address and the CallParticipant are then passed to the SipUtil.sendInvite() method. That method returns a clientTransaction, which the SipTPCCallAgent remembers. The SipTPCCallAgent adds itself as a listener for this call id to the (Jain) SIPServer.

The SipUtil first generates the SDP content. This is partially done by the SdpManager, which in turn gets most of the information it needs from the MediaInfo object.

With this SDP, the SipUtil then populates a bunch of SIP headers (from, to , requestUri, Via-headers, etc), asks its sipProvider for a new call id and a new clientTransaction, and sends the INVITE. The To: field is set to cp.getPhoneNumber().

The MediaInfo object

The com.sun.voip.MediaInfo object is basicly a collection of supported RTP settings. It offers a static findMediaInfo method which can be used to find RTP settings based on a payload byte, or based on encoding, sampleRate and number of Channels.

Adding other parties to the conference

To add other parties to the conference, first select the conference

$telnet_prompt>conferenceId = [some_id]

Then simply type in the phone number of the person you’d like to be added:

$telnet_prompt> phoneNumber = sip:joe@host.com

Complete your request with a newline:

$telnet_prompt> (newline)

Receiving, mixing and sending audio

The following sequence diagram illustrates the process of sending and receiving audio in a conference.

Each conference has a ConferenceReceiver for receiving socket data, which runs in a separate Thread. Attached to each socket is a list of MemberReceivers, corresponding to the parties in conference. This mechanism uses java.nio, each selectionKey corresponds to a MemberReceiver. When data arrives on the socket, the ConferenceReceiver dispatches the data to each memberReceiver’s receive() method, depending on the selectedKeys. Each memberReceiver in turn calls its own receiveMedia method. Eventually the data is stored in memberReceiver.currentContribution variable, waiting to be mixed and sent.

The class diagram below shows the aforementioned classes and their relations.

The central object for mixing audio is the MixManager. It contains an ArrayList of mixDescriptors, which can be added and removed by calling addMix and removeMix. A ConferenceMember calls mixmanager.addMix() to add itself during its initialize() method. This happens when a call has been established (by the SipTPCCallAgent) and we know the port at which the member (CallParticipant) listens for data. At that moment, both the ConferenceMember’s MemberSender and MemberReceiver are also initialized.

Each conference has a ConferenceSender for sending data – but read my remark further down about the loneSender. The ConferenceSender runs as a separate Thread and continuously calls its own sendDataToConferences() method. When a list of all MemberSenders has been constructed, the ConferenceSender calls each member’s sendData() method. That method calls the essential method memberSender.sendData(mixManager.mix()), which causes this ConferenceMember’s MixManager to gather all mixDataSources‘ currentContributions, mix them, after which each memberSender outputs it as an RTP stream to this member’s socket.

It appears that a member is always talking in some whisper group.  For a whisper group with a common mix, a member has a descriptor for the common mix and a descriptor to subtract out its own data. If the member wants to adjust the volume of some other member, then it will have a descriptor for that other member.
For a whisper group without a common mix, a member has descriptors for other members it wants to hear. This is used in Project Wonderland, where an avatar should only hear others in range. If you and I are close enough to each other and not near anybody else, we’d have one descriptor for each other and that’s it.

Questions and answers

Q: Who owns a conference?
A: There’s no concept of an owner.  If you don’t explicitly create a conference, it will be created when the first call joins and destroyed when the last call leaves.  When you explicitly create a conference it stays until someone destroys it.

Q: Suppose Joe and Yvo are in a conference with a bunch of other people, and I want to create a whispergroup for the two of us because we have some offline business to discuss. I guess I should use createWhisperGroup(conferenceId, whisperGroupId) and   addCallToWhisperGroup(whisperGroupId, callId) telnet commands. But how do I know the conferenceId and Yvo’s and Joe’s call ids?
A: If you’re using telnet commands, conferenceInfo (or ci) will display this information. If you’re doing this programatically, you would probably want to specify the callId for each call and you’ll have to pick something for the conferenceId as well.

Q: Is SDP renegotiation supported? Suppose I’m in a conference but I’m turning on my webcam, I’d like to do a new handshake to renegotiate SDP because I can now handle video. Or my broadband connection suddenly deteriorated so i’d like a lower samplerate for my audio. Is something like that possible?
A: I don’t think we ever fully supported re-INVITE.  With project wonderland we have an Audio Quality setting.  That changes the settings and hangs up the call and places the call again.  So we’re managing that at a higher level than the bridge.

Q: What happens exactly when an announcement is played to a conference? Is a treatment “just another call” ?
A: A treatment is an AudioSource so it can be used as the audio source in a MixDescriptor. To play a treatment to a call, the treatment gets added to the ConferenceMember‘s memberTreatments ArrayList. When the next treatment is started, a mix descriptor is added for the member with the treatment as the AudioSource. When the member’s saveContribution method is called, the current treatment’s saveContribution method is called. When it comes time to mix, the mix manager just goes through all the mix descriptors like always. When the treatment finishes, the mix descriptor for the treatment is removed.

Q: How does the integrated STUN server work?
A: We modified the NIST SIP Stack so that it multiplexes the SIP port (usually 5060) with STUN. If data is received on the SIP Port and the first short is 0001 (STUN Binding Request), then the STUN server replies.

Q: What does call migration mean?
A: It’s just a way to transfer a call to another phone without leaving the conference. If I’m at work using my office extension, I can migrate to my cell phone and people in the conference won’t notice except for the change in audio quality.

Points of improvement and other details

The following issues were just some things I noticed while I was investigating the code. Some of them are points for improvement:

• Some data is actually handled by the RequestHandler and not dispatched to the RequestParser, for example the “CANCEL” request, which is dispatched straight to the OutgoingCallHandler or a “DETACH” request, which just closes the socket.
• Some CallHandler methods are static while others are non-static. The non-static methods are invoked by e.g. the RequestParser after looking up the correct CallHandler instance, based on a call id which was passed as a command parameter. Some static methods, however, do exactly the same thing, e.g. recordFromMember.
• There is no decent exception handling in the RequestParser yet. Any exception which occurs while parsing the supplied parameters is caught and rethrown as a ParseException. However, all ParseExceptions are caught on the highest level and ignored, not even logged. This is due to the way the parser works, it’s just a fallthrough mechanism trying to parse commands as the previous one failed.
• The initial INVITE that is sent out by the SipUtil contains a hardcoded CSeq value of 1.
• There is the concept of the loneSender, which means a single ConferenceSender is used to send data to all members of all conferences. Joe: “We tried having a separate sender thread for each conference.  Then we decided it would be better to have one thread set up the work to be done and then fire off as many threads as there are processors to do the the actual sending.  This is the way we’ve been running.”
• SipUtil tries to determine the remoteAddress from the CallParticipant by looking at its phoneNumber. However, the remoteAddress variable is not used!
• The MixManager uses a static WhisperGroup.mixData() method for doing the actual mixing of audio, even when there aren’t actually any whispergroups. A bit confusing.
• I think there is always at least one WhisperGroup: the conferenceWhisperGroup, created when the WGmanager is constructed. A WGManager is constructed when a ConferenceManager is constructed.
• The migrateToBridge request seems to offer cool functionality to migrate a call to another jVoiceBridge instance, but it may not actually work…

Points for further study

If this walkthrough was useful, as a next step you could try and answer the following questions. If you have the answers, please let me know

• Examine threading. How many threads are actually running at any one point in time?
• Event handling. Which event listeners have been defined, how do they work? Which events can be reported and handled inside the bridge?

Quercus is not only an interpreter, it also offers PHP extensions allowing you to call server-side Java code directly from within your PHP script. The Caucho website offers some documentation on how to do that; this opens interesting possibilities on SailFin. You can now use JSR289 functionality directly from within PHP. For example, if you’re running a PHP guestbook, you could extend it to include SIP dial-out functionality.

To deploy Quercus, download the latest version, open the SailFin admin console, select Applications, then Web Applications, then Deploy… and point to the Quercus .war file you just downloaded. If all goes well, your admin console should look something like this:

To verify that your PHP engine is working, point your browser to the default PHP script that comes with the Quercus interpreter: http://localhost:8080/quercus-3.1.6/ (your HTTP port or Quercus version might of course be different). You should get the following screen:

Now go try out some PHP applications on your SailFin server; for example, you could run your own MediaWiki on your local SailFin server! To do so, download the latest MediaWiki distribution and unzip it in the Quercus interpreter root directory (assuming your running the default out-of-the-box domain called domain1):

[SailFin installdir]/domains/domain1/applications/j2ee-modules/quercus-3.1.6/

It’s probably a good idea to create a subdirectory for any PHP applications you try out, so for example:

[SailFin installdir]/domains/domain1/applications/j2ee-modules/quercus-3.1.6/wiki/

And that’s that, you now have a PHP wiki running on SailFin. Try it by pointing your browser to: http://localhost:8080/quercus-3.1.6/wiki/

Of course you will need to complete other steps, depending on the type of PHP application you’re developing or trying out. In the case of MediaWiki, you will need to setup a MySql server and create the appropriate JDBC objects in SailFin. I might describe how to do that in a later blog.