Archive for March, 2016

Real-time text development notes 2

Wednesday, March 9th, 2016

When I started actually trying to use Omnitor’s t.140 library to actually receive some real-time text in my Android app, it seemed like it would be a breeze. They provided a helpful little example program with some extra classes that make it look very easy to use. Nice.

Their library is kind of old, so it is a plugin for the Java Media Framework, which is very old (its website is excited to announce support for MP3!), and plenty of people warn against trying to use JMF on Android. But the cross-platform jar is still available, and it seemed worth a try. Even if it’s not built into the JRE in Android, hopefully we can still use the external jar. However, one problem arose: one of the classes that Omnitor provides implements the javax.media.BufferControl interface, which contains this method:
public Component getControlComponent() {
return null;
}

This is supposed to return a java.awt.Component, but Android doesn’t include this. When people say that “Android doesn’t use Java”, this is what they mean. It is perfectly logical that Android has a different UI layer instead of the desktop windowing stuff. So this doesn’t compile.

But wait, we don’t actually need to do anything window-related here. No one is actually going to use anything this method returns. It returns null, so there’s no way that the object we’re returning actually HAS to provide the functionality of a Component. OK, maybe we can get around this by creating a little placeholder file to silence the compiler:

package java.awt;

public class Component {

}

Upon trying this, I received the best compiler error I have ever encountered:

Ill-advised or mistaken usage of a core class (java.* or javax.*)
when not building a core library.

This is often due to inadvertently including a core library file
in your application's project, when using an IDE (such as
Eclipse). If you are sure you're not intentionally defining a
core class, then this is the most likely explanation of what's
going on.

However, you might actually be trying to define a class in a core
namespace, the source of which you may have taken, for example,
from a non-Android virtual machine project. This will most
assuredly not work. At a minimum, it jeopardizes the
compatibility of your app with future versions of the platform.
It is also often of questionable legality.

If you really intend to build a core library -- which is only
appropriate as part of creating a full virtual machine
distribution, as opposed to compiling an application -- then use
the "--core-library" option to suppress this error message.

If you go ahead and use "--core-library" but are in fact
building an application, then be forewarned that your application
will still fail to build or run, at some point. Please be
prepared for angry customers who find, for example, that your
application ceases to function once they upgrade their operating
system. You will be to blame for this problem.

If you are legitimately using some code that happens to be in a
core package, then the easiest safe alternative you have is to
repackage that code. That is, move the classes in question into
your own package namespace. This means that they will never be in
conflict with core system classes. JarJar is a tool that may help
you in this endeavor. If you find that you cannot do this, then
that is an indication that the path you are on will ultimately
lead to pain, suffering, grief, and lamentation.

This is amazing. It clearly describes the problem, offers understandable suggestions as to the possible causes, provides detailed information about how to fix them, and offers some useful advice/context. This is why programmers need to be good at writing human languages and dealing with people.

The warning here is pretty clear for my case: don’t use JMF, it depends on stuff that isn’t in Android. My situation is definitely what the message is talking about re: grief and lamentation. But…I’m not about to rewrite Omnitor’s whole library. Modifying it not to use JMF sounds pretty hairy. Maybe there’s a way to get past this. How about if we use the actual original version of java.awt.Component? That’s in rt.jar, which comes with your JVM, so we can unzip it and find Component.class, and put that in its own jar. No luck, same error, we’re still trying to use the java.awt package within our project. Maybe try the whole rt.jar? Same problem. Isn’t there some way around this?!

Well, there’s that –core-library option. In Android Studio 1.5, this is found in File > Other Settings > Default Settings > Build, Execution, Deployment > Compiler > Android Compilers. Wow, that’s pretty buried. They sure don’t want me to use this. In fact, if I try to use it, nothing happens. Some StackOverflow posts discuss this flag and how to add it, one of which is similar to an Android bugtracker post that basically says don’t do this. One comment says this no longer works on Android Studio 1.2, which is older than my version. In my great searching, I came upon a mailing list discussion that apparently led to the fantastic error message, but no real help. I am inclined to think that this may now be intentionally disabled, but the vestigial checkbox remains. Shucks.

Let’s back up. The problem is that we need to return a “Component” but the compiler doesn’t know of any such class. If only this interface called for returning something else…aha! Since we are using JMF in a jar file, not as part of the actual JRE, let’s modify that jar. This involves decompiling it, which is no problem in Android Studio (IntelliJ). I decompiled Control.class, copied the source to a new project, and changed
public Component getControlComponent()
to
public Object getControlComponent()
This is sketchy, I know. There is all sorts of risk that other things that use this interface could break. But as far as I know, I’m not using anything else that uses this interface. I just want to use this one little class from Omnitor’s example! So, I unzipped the jar, compiled a new Control.class, replaced the existing .class and repackaged the jar with:
$: jar cf new_jmf.jar *
Plop that into my app as a library, and what do we get?
bad class file magic (cafebabe) or version (0034.0000)
Hm. Dang. Something is wrong with my jar. Looking at the jar in a hex editor, I see that indeed my number is CAFEBABE 00000034, so what’s so bad about that? It was produced by the official jar tool, so it seems legit. Isn’t this what is expected by that error message? Posts online suggested that it had to do with the Java version of your project. I’m using 1.8, so I downloaded the official Oracle JDK 1.7 and set Android Studio to use that for compiling this project. No change. Luckily someone else spent even longer than I did on this and figured out that what this message is expressing (very badly) is that the Java version the JAR was compiled with is bad, since Android only supports 1.7. Fine, I recompiled my modified Control.class again using 1.7, repackaged the jar again, and…it worked! Wow!

Now, simply managing to compile something is not a very big victory. Does the app actually work? I called it from SIPCon1 and sent some text, just trying to write it to the log. Nothing there, except some errors:

03-09 13:47:07.537 8307-8365/com.laserscorpion.rttapp W/System.err: IOException in readRegistry: java.io.InvalidClassException: javax.media.protocol.ContentDescriptor; Incompatible class (SUID): javax.media.protocol.ContentDescriptor: static final long serialVersionUID =-7089681508386434374L; but expected javax.media.protocol.ContentDescriptor: static final long serialVersionUID =912677801388338546L;
03-09 13:47:07.538 8307-8365/com.laserscorpion.rttapp W/System.err: Could not commit protocolPrefixList
03-09 13:47:07.538 8307-8365/com.laserscorpion.rttapp W/System.err: Could not commit contentPrefixList
03-09 13:47:07.540 8307-8365/com.laserscorpion.rttapp I/System.out: No plugins found
03-09 13:47:07.542 8307-8365/com.laserscorpion.rttapp W/System.err: java.lang.reflect.InvocationTargetException
03-09 13:47:07.543 8307-8365/com.laserscorpion.rttapp W/System.err: java.lang.reflect.InvocationTargetException
03-09 13:47:07.544 8307-8365/com.laserscorpion.rttapp W/System.err: java.lang.reflect.InvocationTargetException
03-09 13:47:07.553 8307-8365/com.laserscorpion.rttapp W/System.err: Failed to open log file.
03-09 13:47:07.562 8307-8365/com.laserscorpion.rttapp I/art: Rejecting re-init on previously-failed class java.lang.Class
03-09 13:47:07.562 8307-8365/com.laserscorpion.rttapp I/art: Rejecting re-init on previously-failed class java.lang.Class
03-09 13:47:07.562 8307-8365/com.laserscorpion.rttapp I/art: Rejecting re-init on previously-failed class java.lang.Class
03-09 13:47:07.562 8307-8365/com.laserscorpion.rttapp I/art: Rejecting re-init on previously-failed class java.lang.Class

A series of errors that seems to stem from a failed serialization. It looks like in making changes to jmf.jar, I confused some class about what is what. Something is unsure if the ContentDescriptor in my jmf.jar is the correct ContentDescriptor that it needs, because the serialVersionUID is wrong. This isn’t surprising, since it seems ContentDescriptor doesn’t define an explicit serialVersionUID. The only way around it is to trick the JRE into thinking the new jar is the old one. OK, let’s define what it wants and remake the jar again:

static final long serialVersionUID = 912677801388338546L;

Same problem. Apparently Proguard may mess up your serialVersionUIDs on Android. You can add some options in a Proguard config file (proguard-rules.pro, not proguard.cfg, that’s out of date) to stop this. But this still didn’t help. I got the exact same message, which was strange, because I had changed the value. After banging my head against it for a while, I realized the error message is ambiguous; it didn’t exactly say who expected which value. It found -7089681508386434374L but expected 912677801388338546L, but where did it find and expect those? I still don’t really know, but I don’t care either, because using the other value in my recompiled jar worked:

static final long serialVersionUID =-7089681508386434374L;

Phew, that error went away. But…the rest didn’t. This was not the cause of entire cascade. JMF is still just not working.

This is unsurprising. I was basically warned. I’m firmly in grief and lamentation land, and I knew it was coming. Android is not Java. This old and crusty JMF thing is kind of part of the Java™ platform, but is really not something you can expect to work on Android, which just uses the Java syntax, and a few convenient parts of the Platform™. Some things, like JAIN SIP, use stuff that is basic enough that it’s not hard to make it work on Android. This “media” framework does not. Sigh.

Well, what now? Omnitor’s library depends on this. Android doesn’t even offer a bare RTP framework we can use, since what it has only does audio. This is not great. Time to take stock of my plan.

Real-time text development notes 1

Sunday, March 6th, 2016

As a school project, I’m working on an Android real-time text (RFC 4103) app using the NIST JAIN SIP builds for Android. My professor pointed out that all the problems I was having are probably going to trip up somebody else later, so I should document them. So, internet, here is your first installment of problems I’ve faced that you hopefully won’t. I’m a number of weeks into this project already, so I’ll need to go back and write up everything I’ve already solved. Using JAIN SIP on Android certainly involves some contortions that don’t come up in standard JAIN tutorials.


AsteriskNOW is a handy all-inclusive open-source PBX package (SIP/RTP server). Install it in a VM and you instantly have a SIP server, not too much configuration required. The configuration you do have to do has a bit of a learning curve because it is so full-featured, but overall it seems pretty nice. The help tooltips of the many, many configuration options in the FreePBX web interface leave something to be desired, but I got a couple users working before too long (hint: most of what you need is under Applicatons > Extensions, and you will want to consult Reports > Asterisk Logfiles > fail2ban to see logging of REGISTER attemps).

Because Omnitor’s RTT reference implementation SIPCon1 doesn’t seem to understand 401 and digest authentication, it has trouble registering to a modern server. If there’s some way to force Basic authentication in Asterisk, I’m not sure how. A useful Stackoverflow post by one of the Asterisk developers, Olle E. Johansson, indicated that leaving blank the “secret” for an extension disables authentication entirely for that user, so that works to get SIPCon1 talking to Asterisk.

However, another problem with Asterisk stems from the fact that it is a back-to-back UA. Thanks to Omnitor and Olle E. Johansson (who apparently also happens to be from Sweden), Asterisk can speak T.140 (RFC 4103) in its role as a B2BUA, but a surely unintentional hearing bias shows through its design. As Gunnar Hellström pointed out on the development mailing list in May 2014, Asterisk will immediately drop any call that does not include a negotiated audio stream. I encountered this same issue myself this week when trying to offer a text-only session to SIPCon1 from my own app. Not having used SDP before, it took me a little while to get JAIN SDP to cooperate (more on that later), but eventually I did:

INVITE sip:2000@192.168.1.100 SIP/2.0
Call-ID: dde3cad9c4ad8738870875ba7623d5e9@192.168.1.103
CSeq: 1 INVITE
From: <sip:201@192.168.1.102>;tag=-1455828934
To: <sip:2000@192.168.1.100>
Via: SIP/2.0/UDP 192.168.1.103:5060;branch=z9hG4bK-323734-97510ab0f8dfe28e37a44d62bc8a144c
Max-Forwards: 70
Allow: ACK, BYE, INVITE, OPTIONS, CANCEL
Contact: <sip:201@192.168.1.103:5060>
Expires: 30
Content-Type: application/sdp
Content-Length: 154

v=0
o=201 2122197558 1 IN IP4 192.168.1.103
s=RTT_SDP_v0.1
c=IN IP4 192.168.1.103
t=0 0
m=text 5061 RTP/AVP 100
a=rtpmap:100 t140/1000
a=sendrecv

I think this should be fine. I should also support t140 red, but I’ll worry about that later. I’m pretty sure SIPCon1 would accept this. But Asterisk never gives it the chance, because it assumes every call must have audio. It’s strange that it accepts the call and only then sends an immediate BYE, rather than simply responding 488 Not Acceptable. I think RFC 3261 says that when the original offer is sent with the 200 OK, and the offer is not acceptable, the caller should still send the ACK and then send an immediate BYE, which is similar to what is happening here, but not quite. If Asterisk doesn’t want to accept my text-only session, it should tell me so with 488. Whatever. Luckily Gunnar’s email helped me understand that I’m not crazy and my SDP actually was working correctly.

So Asterisk is out. Instead I tried sipXcom, which is a regular proxy. It’s pretty good so far, but is not without its own problems. Using VMWare Fusion 8.1, I got it running fairly quickly, faster than Asterisk, but ran into some quirks. It basically assumes you are using hardware phones, and when setting up a new extension, the only way around this is to say your new phone is “Jitsi”. In fact I have been using Jitsi for testing INVITEs, but to claim that all softphones must be Jitsi is a little silly. Anyway, a bigger problem is that after suspending and restoring the VM, all attempts to register got an immediate 408 Request Timeout. How could you issue 408 immediately? My Expires: header was 600, and the only 600 that could have passed in that time was maybe 600 microseconds. Rebooting fixed it.

The final problem with sipXcon for my purposes is that it doesn’t allow you to leave a user’s password blank, or otherwise disable authentication. This is a totally sensible security feature, but in this rare case, I actually do want the freedom to do this. Or at least, I’d like to use Basic authentication instead, since SIPCon1 can probably manage that. It might be possible to do at least one of these by editing a configuration file, but the web interface doesn’t seem to give an easy way. I’ll look into that later. For now I can call my app from SIPCon1, with SIPCon1 still connected to Asterisk and my app going through sipXcom.