Content from 2017-06

Lately I've been thinking again about some way to unify (my) usage of different communication channels. Part of that is the additional distraction and lack of ease of use for some of the applications I'm forced to use.

This is partially a feature of my habits, i.e. I'm not a mobile phone user. At all. But "apps" like WhatsApp, WeChat, while thankfully having web clients, still force me to use a comparatively clunky interface.

Then we have Slack, the IRC clone that will eat every other IRC clone there is. Or maybe not. In my case it's the primary business-related communication form, after email.

Good that I'm not using Facebook, the moloch, but I imagine for a lot of people it's the primary way to use the web and internet. I haven't researched Facebook integration at all though, so there might be ways of integrating it more or less easily.

The previously mentioned channels were all active ones, but there's also a lot of passive consumption going on. (RSS and blogs, forums,) reddit, Hacker News are all channels that I frequently use. In case of reddit and Hacker News of course there's is the active element of posting, commenting, voting, but I rarely do that, so they fall under passive for me too.

So again, why unification? For all the above, getting notified (if new content is available) is a pain, comparatively. Both in the sense that for some of them (chat) the threshold is quite low, so reacting in near real-time is important, while for others it's absolutely the other way round, even though I'm still habitually checking them in case I'm bored (see how that goes?).

Unifying them would allow to aggregate, combine, filter in a general fashion instead of having (or not having in most cases) a distinct way to do that for each channel.

So again, would that solve the problem? I'm doubtful. On the one hand, there's clearly a need to remove friction. On the other hand, the cost of implementing it, the lack of distinctive features for each channel (visual mostly) would also undermine some of information. Possibly only at the start, it's hard to tell. I can however say that using RSS readers for me never worked out, precisely because the visual appearance is such a strong discriminator to (not) consume content. Though rtv, a console-based reddit client, worked rather well for some highly text-based content.

What other considerations are there? Well, the split between different contexts would be one thing. There's at least the work/life split for me, if not for many others.

Fidelity, as in, most text-based content can be viewed on the console, even if it might look better in a different renderer (browser). Showing pictures/clips is difficult on the console, but there are ugly hacks that can work around that problem if absolutely necessary (I'm personally not a fan).

Amount, blogs are rather infrequent, but have lots of text per each post, chat is very high frequent comparatively, but only has a few words per "post".

Context, again, there's also different groups in the personal context, that is, e.g. family, friends, different hobbies and interests, with each group having a somewhat overlapping set of sources.

So again, what can be solved? Technically, at least getting more sources into a single format is achievable. There are bridges from Slack to IRC, from RSS to IRC, etc. I'm choosing IRC here because it's a form of lowest common denominator, but similarly it could be mapped to email too. While IRC isn't good for long-form content, it can contain links which can then be viewed in other renderers, solving the notification issue. (Well, you still need to pay attention to the IRC client. In my case I'm always online on a VPS, so I need still to pass through notifications from the IRC client to the current local machine.)

What options would a unified architecture give us? E.g. having a single feed for chat, email, blog posts etc. for a group of people (channels). This can again be achieved manually, by tying in bots to post on behalf of a user, though in the architecture of IRC it wouldn't make sense to post some of these things publically - it's "your" view of the conversation, not the public view. That is, you'd want to view a feed with incoming emails, blog posts (Twitter, what have you) from a person inline.

Now, inertia. Given how XMPP basically failed and how each platform provider is aggressively trying to get people into their walled garden, what chance is there for a standard here?

Apart from that, can this idea be implemented purely client-side? AFAIK yes, there's still friction with the different technologies being integrated, but as a central communication hub this would still make sense.

Building on top I have some further (obvious) extensions in mind, the usual spam filters, deduplication, aggregation/search, also everything statistics basically, that can be applied on top.

Different interfaces would be available to have a view on the streams, or groups of streams. Traditionally this all hasn't worked out I feel, with the exception of very, very narrow things like email and text-based chat there's just a lot of variation going on.

How would this look like? For me, one console window to show everything, with desktop notifications on top. For others, the same in a browser perhaps, or (take a deep breath) a native application instead.

In any case, food for thought. I'm hoping to follow up on this with a more focused design at some point.

Just as a quick note, JSS and the JAVA package too won't allow you to treat LispObjects objects as JavaObjects for the purposes of JFIELD and JSS:GET-JAVA-FIELD. But if you still want to access internal fields and implementation details (the usual warnings apply!), try the following:

(jss:get-java-field (jss:new 'JavaObject (car (threads:mapcar-threads #'identity))) "javaThread" T)

This was because I was looking for an answer to a question on #abcl, but even then, wrapping the "Lisp" object in a JavaObject manually helps achieve the requested goal of retrieving the Java Thread object.

The T at the end is necessary here because the field is actually package-private - but that's to be expected if you want to access internals without an (official) API.

A feature of JVM byte code that I knew about, but didn't concern me too much when I previously hacked on ABCL was interesting to see in a particular bug report, where a function, when compiled, got rejected by the byte code validator:

(compile NIL (lambda (list) (nth (lambda ()) list)))
;; => Compiled function can't be loaded

This is true for any class loader, so COMPILE-FILE with LOAD would show the same behaviour.

The reason here is inlining: In order to give better performance when some information about the types is known, the call to NTH will actually be optimised here. One option to see this is to set a debug option to dump the generated byte code while the compilation runs:

(setf jvm::*compiler-debug* T)

(Part) of the output will look like this:

  0 GETSTATIC      #24 <Field LispObject 42c1_92be_e5b48c1894c6.LFUN1812724>  
  1 ALOAD          (1)  
  2 SWAP             
  3 INVOKEVIRTUAL  #30 <Method LispObject LispObject.NTH(int)> -1 
  4 ARETURN          

Using DISASSEMBLE is not an option precisely because the byte code can't be loaded at all (that's actually a nice idea as another function to dump not only the compiled FASL content, but also disassemble the contained byte code too).

With that option set (and perhaps setting a few BREAK statements in the compiler), it's somewhat easy to debug the compiled byte code and to see that the validator notices that the Java method that implements the NTH function requires an integer (fixnum) argument, but the LispObject (the lambda) doesn't match that signature. This is not a problem if this was a regular call. In fact, with inlining disabled the NTH function will still raise an error for an argument with the wrong type!

Finally the fix is to check for the derived type in the "P2" transformation function for NTH, COMPILE-NTH:

(define-inlined-function compile-nth (form target representation)
  ((check-arg-count form 2))
  (let* ((index-form (second form))
         (list-form (third form))

    ;;; new check here
         (index-type (derive-compiler-type index-form)))
    (unless (fixnum-type-p index-type)
      (compile-function-call form target representation)
      (return-from compile-nth))
    ;;; till here

    (with-operand-accumulation
        ((compile-operand index-form :int)
         (compile-operand list-form nil)
         (maybe-emit-clear-values index-form list-form))
      (emit 'swap)
      (emit-invokevirtual +lisp-object+ "NTH" '(:int) +lisp-object+))
    (fix-boxing representation nil) ; FIXME use derived result type
    (emit-move-from-stack target representation)))

Note the falling back to a "general" function call using COMPILE-FUNCTION-CALL here in case the type is not known in advance, or not a fixnum type (though that could also raise a warning here already).

Again, compiling the above function again looks a bit different in the general case:

  0 GETSTATIC      #29 <Field Symbol 4faf_99d9_62a0381d4d65.SYM1814566>  
  1 GETSTATIC      #33 <Field LispObject 4faf_99d9_62a0381d4d65.LFUN1814565>  
  2 ALOAD          (1)  
  3 INVOKEVIRTUAL  #39 <Method LispObject LispObject.execute(LispObject,LispObject)> -2 
  4 ARETURN          

If instead a fixnum constant is used, (nth 1 list), this simplifies a lot:

  0 ICONST_1         
  1 ALOAD          (1)  
  2 SWAP             
  3 INVOKEVIRTUAL  #24 <Method LispObject LispObject.NTH(int)> -1 
  4 ARETURN          

Compare that to adding a (declare (type fixnum ...)) declaration - not as good as a constant argument, but still directly calling the Java method:

  0 ALOAD          (1)  
  1 INVOKEVIRTUAL  #24 <Method int LispObject.intValue()> 0 
  2 ISTORE         (1)  
  3 ILOAD          (1)  
  4 ALOAD          (2)  
  5 SWAP             
  6 INVOKEVIRTUAL  #28 <Method LispObject LispObject.NTH(int)> -1 
  7 ARETURN          

Note here that the type error (e.g. for again supplying the (lambda ())) will be raised at the caller site already!

Lastly, a good idea would also be to generally add more hints, as e.g. SBCL does, to debug other issues ("failed to inline because ..."), but that's for another day.