Byte code verification and ABCL

A feature of JVM byte code that I knew about, but didn't concern me too much when I previously hacked on ABCL was interesting to see in a particular bug report, where a function, when compiled, got rejected by the byte code validator:

(compile NIL (lambda (list) (nth (lambda ()) list)))
;; => Compiled function can't be loaded

This is true for any class loader, so COMPILE-FILE with LOAD would show the same behaviour.

The reason here is inlining: In order to give better performance when some information about the types is known, the call to NTH will actually be optimised here. One option to see this is to set a debug option to dump the generated byte code while the compilation runs:

(setf jvm::*compiler-debug* T)

(Part) of the output will look like this:

  0 GETSTATIC      #24 <Field LispObject 42c1_92be_e5b48c1894c6.LFUN1812724>  
  1 ALOAD          (1)  
  2 SWAP             
  3 INVOKEVIRTUAL  #30 <Method LispObject LispObject.NTH(int)> -1 
  4 ARETURN          

Using DISASSEMBLE is not an option precisely because the byte code can't be loaded at all (that's actually a nice idea as another function to dump not only the compiled FASL content, but also disassemble the contained byte code too).

With that option set (and perhaps setting a few BREAK statements in the compiler), it's somewhat easy to debug the compiled byte code and to see that the validator notices that the Java method that implements the NTH function requires an integer (fixnum) argument, but the LispObject (the lambda) doesn't match that signature. This is not a problem if this was a regular call. In fact, with inlining disabled the NTH function will still raise an error for an argument with the wrong type!

Finally the fix is to check for the derived type in the "P2" transformation function for NTH, COMPILE-NTH:

(define-inlined-function compile-nth (form target representation)
  ((check-arg-count form 2))
  (let* ((index-form (second form))
         (list-form (third form))

    ;;; new check here
         (index-type (derive-compiler-type index-form)))
    (unless (fixnum-type-p index-type)
      (compile-function-call form target representation)
      (return-from compile-nth))
    ;;; till here

    (with-operand-accumulation
        ((compile-operand index-form :int)
         (compile-operand list-form nil)
         (maybe-emit-clear-values index-form list-form))
      (emit 'swap)
      (emit-invokevirtual +lisp-object+ "NTH" '(:int) +lisp-object+))
    (fix-boxing representation nil) ; FIXME use derived result type
    (emit-move-from-stack target representation)))

Note the falling back to a "general" function call using COMPILE-FUNCTION-CALL here in case the type is not known in advance, or not a fixnum type (though that could also raise a warning here already).

Again, compiling the above function again looks a bit different in the general case:

  0 GETSTATIC      #29 <Field Symbol 4faf_99d9_62a0381d4d65.SYM1814566>  
  1 GETSTATIC      #33 <Field LispObject 4faf_99d9_62a0381d4d65.LFUN1814565>  
  2 ALOAD          (1)  
  3 INVOKEVIRTUAL  #39 <Method LispObject LispObject.execute(LispObject,LispObject)> -2 
  4 ARETURN          

If instead a fixnum constant is used, (nth 1 list), this simplifies a lot:

  0 ICONST_1         
  1 ALOAD          (1)  
  2 SWAP             
  3 INVOKEVIRTUAL  #24 <Method LispObject LispObject.NTH(int)> -1 
  4 ARETURN          

Compare that to adding a (declare (type fixnum ...)) declaration - not as good as a constant argument, but still directly calling the Java method:

  0 ALOAD          (1)  
  1 INVOKEVIRTUAL  #24 <Method int LispObject.intValue()> 0 
  2 ISTORE         (1)  
  3 ILOAD          (1)  
  4 ALOAD          (2)  
  5 SWAP             
  6 INVOKEVIRTUAL  #28 <Method LispObject LispObject.NTH(int)> -1 
  7 ARETURN          

Note here that the type error (e.g. for again supplying the (lambda ())) will be raised at the caller site already!

Lastly, a good idea would also be to generally add more hints, as e.g. SBCL does, to debug other issues ("failed to inline because ..."), but that's for another day.