Tackling Core Audio

I've been working with Alex Wiltchko's Novocaine, which works nicely as a pain-reducing wrapper for lots of basic core audio stuff. Unfortunately, it's not idiot-proof, and I've made at least one idiot-level mistake with it.

Alex very nicely packaged up an AudioFileWriter class with the project which works great for me in the simulator, but fails unpredictably when I try to use it on a device. So for me, anyway, it seems that dream of avoiding touching core audio, and the dreaded AudioStreamBasicDescription has not arrived.

Today I'm playing around with ExtAudioFileWriteAsync, seeing which sorts of settings will work for me.

Writing to AIFF is working without too much trouble, but I'm having a heck of a time writing to m4a. It looks as if ExtAudioFileWriteAsync is sensitive about the number of frames passed when writing files of format kAudioFormatMPEG4AAC. When passing a buffer of 10,000 frames, I get a crash with this error code: -66567.

For anyone reading this who can't make sense of these error codes (like me, until yesterday), you can search the headers of whichever framework you think the error originated from for the code. Here's some explanation.

Anyway, -66567 resolves to kExtAudioFileError_MaxPacketSizeUnknown. I'm unable to find any documentation indicating just what this message means.

Now, when I reduce the number of frames I'm passing to ExtAudioFileWriteAsync to 512, the error message goes away, but the resulting file is only 28 bytes long and is unplayable. This is only in on my iPad. All works fine in the simulator.

Now I'm simplifying and trying to work with the synchronous version of ExtAudioFileWrite. This is failing immediately with kExtAudioFileError_MaxPacketSizeUnknown, even when I give it a very small number of frames (100).

This StackOverflow post indicates that there are some properties that are required to be set. I'm not sure that I read the documentation the same way this poster does, however it seems that in their case, setting kExtAudioFileProperty_CodecManufacturer solved a problem.

Spent four hours on FilePlay today. This app is going to involve quite a lot of hard work. My current status is this: FilePlay successfully streams multiple files from disk. On the iPhone it can handle about three stereo files. Not very impressive. Discovered the files Jay gave me were 24-bit. So I'll be able to stream more like 4 files easily. Still not super impressive. Next step is to be able to swap out one file for another mid-play. Tried doing this and am getting errors in fseek. Blowing off yoga in favor of programming. Went for a 25 minute walk today though.

Cheap and Deep app progress

Spent most of the day today working on FilePlay, a class that's part of EZPlug and streams files from disk and into an EZPlug network. It's just a little bit trickier than I can easily wrap my head around. Running in to multiple problems: memory issues with no warnings in XCode, and niggly little logic bugs of the type that crop up when I work with juggly-type code. Somehow, anything involving getting to the end of something and then jumping back to the beginning always gives me trouble. I'm going back and forth a bit about whether I should just use Apple's code, because surely they know what they're doing, and surely I don't. But generally, I'm settling on the idea that FilePlay is something I'll be using again and again, and it's worth taking the time to get it right, and to make sure that any time I want to do this stuff in the future, it will be very easy. Very E-Z.

debugging C++ in XCode

I've been frustrated by how XCode's debugger seems to be easily confused by C++. Also generally annoyed that it's hard to view the contents of an array by unfolding arrows. Things are moving along much more nicely now that I've started paying more attention to the command-line interface to GDB, the debugging tool beneath xcode's debugger gui.

For example, when trying to inspect a variable, I was getting this error: "current stack frame does not contain a variable named `self'. The problem was that GDB thought it was looking at ojective-c when in fact it was C++. The fix was to manually tell GDB via the console that it's looking at C++.

set language c++

The GDB command line interface is also really nice for examining buffers. When working with audio, you get deal with huge sets of data, tens of thousands of samples. There's no obvious way to examine that data via the debugger ui. Even harder, what if you want to examine a small chunk of it somewhere in the middle. With the GDB console, I can type this:

(gdb) x/64f sample.data_ + sampleReadPosition

Where sample.data_ is my audio data and sampleReadPosition is where I want to start looking. A few more things about this command: x is the main command which inspects data. /64 means repeat 64 times, and format as a float. So this command says: look at the memory pointed to by sample.data, jump forward by sampleReadPosition step, and format it as a float.

Here's the output.

0xbd9b458: 0.0132450331 0.0792260543 0.0219122898 0.0856349394

0xbd9b468: 0.0309457686 0.0916776061 0.0396130271 0.0969878212

0xbd9b478: 0.0487075411 0.101870783 0.0573747978 0.106387526

0xbd9b488: 0.0656758323 0.110171817 0.0739768669 0.113162637

0xbd9b498: 0.0818506405 0.115421005 0.0890530124 0.116946928

0xbd9b4a8: 0.0958281234 0.117313154 0.102237009 0.117313154

0xbd9b4b8: 0.107913449 0.116214484 0.112796411 0.114322335

0xbd9b4c8: 0.116946928 0.111270487 0.119998783 0.107547231

0xbd9b4d8: 0.122623369 0.102969453 0.1241493 0.0977202654

0xbd9b4e8: 0.124515519 0.0916776061 0.1241493 0.0845362693

0xbd9b4f8: 0.122989595 0.076967679 0.120731227 0.0686666444

0xbd9b508: 0.117313154 0.0596331693 0.11358989 0.0505386516

0xbd9b518: 0.108279675 0.0404065065 0.102603227 0.0302133244

0xbd9b528: 0.0961943418 0.0200201422 0.0886867866 0.00909451582

0xbd9b538: 0.0807519779 -0.00146488845 0.0724509433 -0.0124515519

0xbd9b548: 0.0634174645 -0.0230109561 0.0539567247 -0.0335703604

The GDB manual is here.

If you find yourself working outside the cocoa frameworks (and perhaps even if you don't), I highly recommend digging in to GDB.

LibPD with Thicket

Developing music software for iOS has been a thrill and a terrific frustration. I've been looking for weeks for an audio engine which could manage signal chains efficiently for me, has a license which won't force me to open-source my software, and can be configured in a c-flavor language. I've come up with basically nothing. My only option (and I'm very grateful for this option) seems to be libpd -- an embeddable version of pure data. This is great, except for the fact that PD designed as a graphical programming language, and to someone like me accustomed to writing code, strikes me as bizarre, wrong, difficult, painful. But I'm going to stick it out for a while and see if PD's beauty decides to present itself to me.

I'm not confident about my ability to write an efficient audio engine, but I am very confident about my ability to use code to express artistic and structural concepts. So I'll be using PD only for signal chain, keeping absolutely as much application logic as possible in code. This brings me to the question of which language to use. Of course, the iPHone's native language is objective-c. But I'm liking C++ a lot more. The syntax is simpler. And it allows me to write all of my code in the header files if I feel like it. I know this is considered bad form, but sometimes I like to take off my engineer's hat and put my artist hat on. With my artist hat, I'm allowed to look at a technique, and say "too slow, too many keystrokes". And that's how I'm feeling about Objective-C vs C++. So I'm tending to thing that for audio logic, C++ is the way to go.

Mixing languages on the iPhone is a pain in the ass. You can mix Objective-C and C++, but you have to jump through hoops in order to avoid compiling your entire application as Objective-C++. That can be necessary if you want to mix C and Objective-C in other parts of the application. I'm currently getting an inexplicable linker error in XCode: "libpd_bang(char const*)", referenced from:". This is maddening, especially to a relative newb. The complier doesn't complain, but the linker fails. I've come across a similar problem before, but I don't remember what the solution was. I'll post back here if I figure it out.

EDIT

I fount my previous linker error. It's described here:

http://stackoverflow.com/questions/4812183/xcode-is-not-even-trying-to-compile-some-of-my-mm-files-then-fails-while-link

This one is not the same however. The previous problem was that the file wasn't even being compile, because it wasn't added to the target. In this case, the file is being compiled. The function in error is even being called successfully from PDBase.m, but it's not being called successfully from other files.

EDIT

Ok. I think I've solved it. Using the "extern c" as recommended here helped:

http://iptrk.ionismus.de/post/94485387/mixing-ansi-c-and-c-in-a-xcode-3-x-project

EDIT

PD's patching graphical interface was driving me crazy. I've decided to to just become a better programmer and stick to low-level stuff. It's not that much more work in the end, and I get to keep my hands on the keys, rather than reaching for the mouse.

Ajax and Aspect-Oriented Programming

I'm working on an ajax application which contains a number of different widgets which can manipulate data. In many cases, multiple widgets depend on the same data. I've been thinking of ways to notify all relevant widgets when a piece of data changes, to let them know that they need to fetch new data and refresh themselves.

Related to this is the question of caching. If widgetA fetches data from the server, and widgetB depends on the same data, widgetB should be able to access the data that widgetA just fetched.

So what I'm moving toward is this:

A single "service" object which contains ajax accessor methods.

A "cache" wrapper around the service object which intercepts calls, decides if cached data can be used, and also decides when listeners need to be notified of new data.

Here's my Cache object so far.

Cache = function(options){
  var self = {};
  var dataSource = options.dataSource;
 var log = UNAB.Debug.NamedLogger("UNAB.Cache");
 var aopCallbacks = {};
  
  var aopCallbackCheck = function(name){
    log.debug("aopCallbackCheck called for: " + name);
    if(aopCallbacks[name]){
      aopCallbacks[name]();
    }
  }
  
  $jq.each(options.dataSource, function(key){
    self[key] = function(){
      // wrap callbacks with a function which will invoke an aop handler
      var processedArguments = UNAB.Util.makeArray(arguments, function(arg){
        if(typeof arg === "function"){
          return function(){
            arg.apply(null, arguments);
            aopCallbackCheck(key);
          }
        }else{
          return arg;
        }
      })
      dataSource[key].apply(null, processedArguments);
    };
    // add a "Cached" method to the object, which will just pass values straight through,
    // possibly use the cache, and not invoke any aop stuff
    self[key + "Cached"] = dataSource[key];
  });
  
  self.addAopCallback = function(options){
    aopCallbacks[options.name] = options.fn;
  }

  return self;
}

How it works is this:

You pass your data accessor object to the cache in the constructor:

wrappedDataServiceObject = Cache({dataSource: dataServiceObject})

then you can call all of the methods that were available on dataServiceObject on wrappedDataServiceObject. But wrappedDataServiceObject has some addional stuff. For every method xxx, wrappedDataServiceObject adds a method xxxCached. I haven't worked out just what I'm going to do with this. For now method xxx does the same thing as method xxxCached.

Additionally, you can register listeners which get run whenever a non-cached method's callback gets called. The callback is wrapped with a new function which calls the callback, but also looks for any aopCallbacks you've registered. This way, you can broadcast to all listeners that new data has been checked. Note -- this happens only on method xxx, not on method xxxCached.

 wrappedDataServiceObject.addAopCallback({
   name:"deleteContact",
   fn: function(){
     $jq(".contactsChangedListener").trigger("contactsChanged");
   }
 })

Things in my app were beginning to look a little spaghetti-ish, and I was getting weird endless loop problems. Hopefully organizing stuff this way will make things easier to manage.

Bundling multiple Asyncronous requests

I'm working on a project which involves gathering multiple little bits of data via separate ajax calls. For example, I make one call to get a list of "groups", then another call to get details for each of those groups.

What I have been doing is loading the list first, displaying some information on the page, then filling in more information as the subsequent details calls come back.

This creates a bit of flicker and annoyance as the details on each item get filled in. I've decided that I want to try fetching all of the data at once, and waiting until it's all collected until I do anything. So I've created a function which allows me to submit a back of function references, along with their arguments. If one of the arguments is a callback, I wrap it with another function which allows me to tell if it's been called or not. After all of the callbacks have been called, I call the master callback for the bundle.

Here's the code.

 
  $jq = jQuery;

  bundledAsync = function(options){
    var callbacksRemaining = 0;
    var decrimentcallbacksRemaining = function(){
      if(--callbacksRemaining == 0 && options.bundleCallback){
        options.bundleCallback();
      }
    }
    // Look through the args searching for functions.
    // When one is found, wrap it with our own function so 
    // that we can keep track of which callbacks have returned
    // this assumes that each callback is only called once
    $jq.each(options.calls, function(index, call){
      $jq.each(call.args, function(index, arg){
        if(typeof arg === "function"){
          callbacksRemaining++;
          call.args[index] = function(){
            decrimentcallbacksRemaining();
            arg.apply(null, arguments);
          }
        }
      });
    });
    // now actually call all the functions
    $jq.each(options.calls, function(index, call){
      call.fn.apply(null, call.args);
    });
  }

And here's how your run it:

 bundledAsync({
    calls:[
      {
        fn: settings.service.getGroupsCached,
        args: [1234234234,  function(resp){}]
      }
    ],
    bundleCallback: function(){}
  })

///////////////////////////// EDIT ////////////////////

After giving this a try, I realized that it wasn't exactly what I needed. I needed to not only bundle async functions, but to allow the return of certain functions to spawn others. So now I have this:

  bundledAsync = function(options){
    
    var callbacksLeft = 0;
    
    var decrimentCallbacksLeft = function(){
      if(--callbacksLeft == 0 && options.bundleCallback){
        options.bundleCallback();
      }
    }
    
    var doCalls = function(calls){
      $jq.each(calls, function(index, call){call.fn.apply(null, call.args)});
    }
    
    // // Look through the args searching for functions.
    // // When one is found, wrap it with our own function.
    // // This assumes that each function has exactly one 
    // // callback, and that each callback is called exactly once
    var wrapCallbacks = function(calls){
      $jq.each(calls, function(index, call){
        $jq.each(call.args, function(index, arg){
          if(typeof arg === "function"){
            callbacksLeft++
            call.args[index] = function(){
              arg.apply(null, arguments); // call the original callback
              if(call.calls){
                // maybe we don't want to create the child calls until after
                // the parent has returned. In that case, pass a function instead of an array
                if(typeof call.calls === "function"){ 
                  call.calls = call.calls();
                }
                wrapCallbacks(call.calls);
                doCalls(call.calls);
              }
              decrimentCallbacksLeft();
            }
          }
        });
      });  
    }
    wrapCallbacks(options.calls);
    doCalls(options.calls);
  }
  

and an invocation that looks like this:

  
  service.bundledAsync({
    calls:[
      {
        fn:  settings.service.getGroupsCached,
        args: [
          function(listsArg){
            listsSummary = listsArg;
          }
        ],
        // These are child function calls of settings.service.getGroupsCached
        // I want to wait until getGroupsCached returns before I create the 
        // child calls, b/c I won't know what the child calls are until
        // getGroupsCached returns
        calls: function(){return makeArray(listsSummary, function(list){
          return {
            fn: service.getGroupCached,
            args: [list.id, function(resp){listsDetail.push(resp)}]
          }
        })}
      },
      {
        fn: settings.service.getAllContactsCached,
        args:[function(resp){
          contacts = resp;
        }]
      }
    ],
    bundleCallback: function(){
      lists = listsDetail;
      view = UNAB.TemplateRenderer.replaceHTMLResults(
        settings.container,
        "PrivateListBox",
        {lists: lists}
      ) 
    }
  });
  
  

The syntax could be lovelier, but I like how I can describe a tree of dependant asyncronous function calls with this. Haven't tested it too much yet, but it seems to be working.