Saturday, July 16, 2016

J provokes self-talk (5)

On with sorting...

I was totally stunned when the |:/:~|: worked yesterday - I would have sworn that I'd tried that (and 100 frustrated variations of it) unsuccessfully before I started writing, and I was only planning to introduce it as a segue to partition sorting. I'll postpone that while I explore this unexpected success.

I haven't found the documentation yet, but it seems irrefutably true that, when sorting a 'matrix' (less typing) by column, J performs tiebreaking via adjacent right values. By controlling the sort order, I can sort my hand suitably. The verb index i. manages the sort order, when used as a dyad, by converting the values into numeric indices, which can then be sorted in the usual numeric sequence. (Is it my imagination, or is that pretty cute?)

'AKQJT98765432SHDC' i. |:unsrtd
16 6
13 8
13 3
16 12
14 11
13 12
16 4
14 1
15 3
13 4
14 12
14 6

14 9

Grade that, and apply to the original, and transpose:

|:|: unsrtd) /: 'AKQJT98765432SHDC' i. |: unsrtd
SSSSHHHHHDCCC

JT62 K8 5 3 2 J T8 2

The parentheses are required.

In terms of my main task, which is dealing hands for analysis, that's all I need. I just have to iterate the process across the four hands, which I don't think will be too hard, and format the output into a file somewhere for now. Later I'll look at calling a DLL and handling the results, as well.

Rather than press on, I'm going to pause to write up my partition sort discovery. Otherwise I'll forget. Originally, I was sorting the suits and the values separately - I don't, in retrospect, know why - so I had:

\:~ {"2 unsrtd
SSSSHHHHHDCCC

and

d2 =.  (1 {"2 unsrtd ) \:{"2 unsrtd
6J2T3K285J82T

so, with laminate:

SSSSHHHHHDCCC
6J 2T3 K2 8 5 J 8 2T

It took a lot of pain to find the key verb for this process, which turned out to be 'cut' ;. 'Cut' is not a verb, it's an adverb, in point of fact, which says: apply the associated verb to y over partitions defined by a boolean list x. If I could create a boolean list 1000100001100 to represent the suit lengths and apply a sort verb to the partitioned card values, I'd be right.

Thanks to Alvord & Thomson, I had an idea about how to do this. There is a verb 'rotate' |. (they use it to test for duplicates) which gives:

_1 |. SSSSHHHHHDCCC
CSSSSHHHHHDCC

rotating 1 to the right via the _1 index. By testing for the Boolean not-equal ~: I derive my boolean list of distributions.

SSSSHHHHHDCCC

CSSSSHHHHHDCC
1 000 10 0 0 0 11 0 0

Using box < to demonstrate:

1 0 0 0 1 0 0 0 0 1 1 0 0 <;.1 d2
┌────┬─────┬─┬───┐
│6J2T   │   3K285│  J│82T  │
└────┴─────┴─┴───┘The 1 in the cut says to start the partition with the 1's in the list.

I want the suits to be sorted in car order, so I'm going to write a verb (otherwise the sentence gets too hard to read) called ssrt.

1 0 0 0 1 0 0 0 0 1 1 0 0 ssrt ;.1 d2
JT62
K8532
J

T82

Woot!
1 0 0 0 1 0 0 0 0 1 1 0 0 < @ ssrt ;.1 d2
┌────┬─────┬─┬───┐
│JT62   │   K8532│  J│  T82│

└────┴─────┴─┴───┘
Introducing the conjunction @ which allows J to execute 2 verbs more or less simultaneously (there are degrees of simultaneity; another time)

Finishing off for now, the verb ssrt:

ssrt =:  /: 'AKQJT98765432' & i.

This is using a style of programming J calls 'tacit', due to the absence of apparent variables. Of course the variables are y and x, as always, and the challenge is to manage them. Inside a 'subroutine' the order of operations isn't quit the same as in open code; in fact it's the same as inside parentheses in open code. This matters quite a lot, because of the strictures of data handling.

Take ssrt. It contains 2 verbs, grade and index. We want the x of grade to be the y of index and J provides this via the 'hook' facility, which is just a name for this setup. The y of index will be the y of ssrt. (If ssrt had an x, this would be the x of grade, but in its absence, ssrt's y gets re-used. Can I stop that from happening? I don't know.)

Now, index also requires an x , which in this case is a constant, our suit rank order, and in this case the constant has to be bonded & with the verb.

At this moment of my life, nothing can persuade me that this isn't a mess. So far, a navigable mess, but one that seems...inelegant.

J provokes self-talk (4)

OK, so back to the main game. I've managed to create something that looks vaguely like a hand of cards.
┌─────────────┐
│CSSCHSCHDSHHH   │
│8 6J 2 32T KJ T2 8 5   │

└─────────────┘
I'll sort the formatting out later. I mentioned earlier that Volker & Thomson clearly aren't card players; who ever played with an unsorted hand? Never mind, sorting (/:~ ) is one of my favourite things.

< /: unsrtd
┌─────────────┐
│8 6J 2 32T KJ T2 8 5   │
│CSSCHSCHDSHHH   │

└─────────────┘
This clearly is not much use. What seems to have happened is that we've sorted each 'column'. Well, maybe I can transpose (|:) and sort...

< |: \: ~ |: unsrtd
┌─────────────┐
│SSSSHHHHHDCCC   │
│TJ6 2K8 5 3 2 J T8 2   │

└─────────────┘
...and transpose back. Great! Well, more or less. Have to work out what's going on with that STJ in a moment. Meantime, what's happening with that sort statement? It's very interesting. /: is called 'grade', and what it outputs, when applied to one input, is a list, in sorted order, of the original positions - the indices - of the data to be sorted. Eg:

 n_suits
CSSCHSCHDSHHH
/: n_suits
0 3 6 8 4 7 10 11 12 1 2 5 9      

You can see the 0th, 3rd, 6th elements are the CCC of the sorted list. This list of indices can be applied to any other list, but for now at least, it's enough to apply it to itself:

n_suits /: n_suits
CCCDHHHHHSSSS                

Or, as above, use the shortcut ~ which says, in effect, use the y input additionally as x input. This also illustrates another feature of J - the fact that a verb (function, operator,...) can have two different  functions, depending on whether it has one input (a monad) or two (a dyad). (Note: /: = grade down \: = grade up)

It seems very fortunate to me that the suit values are so nicely sorted internal to the suits. I mean, in the original list the order of the C was 8,T,2. Why when relocating the C did J relocate them to positions 1,0,2? The order, in itself, of the S is easy to understand; since we're descending (to get SHDC), the T comes before the J - but it seems too clever by half to decide to partition sort without being asked. Is this always going to happen? Should I test it, ransack the documentation, or work out a partition sort? Plus, I need to have a custom sequence...

I'm going to work these two functions out, because I really can't believe the good fortune of that list sort. Maybe I'll find the documentation serendipitously.

Friday, July 15, 2016

J provokes self-talk (3)

I can't really postpone talking about J's data structure(s). Inevitably, they are crucial. As I start to understand them, they seem fairly simple - but I've beaten my head against a few walls to get this far. Starting with 0{t12, which returns a row of an array, and (13|t1){'AKQJT98765432', which returns characters from a string. Que?

Lists are the (maybe 'a') fundamental data structure. They are so fundamental that for a numeric list, the sole delimiter is a blank space.  2 3 6 7 is a list. It takes a while to get used to this, if like me you come from a universe of list = ({[,,,,...}}). I'm not criticising - I'm impressed by the commitment to terseness. But it does take a while. Also, a string is a list (of characters) and likewise an array is a list (of lists). Lists have shape ($).

$ 'string'
6                  The shape of a string (or any one dimensional list) is its length.
$ t11

4 13             The shape of a list of lists (do NOT think about arrays) is rows * columns.

{ is called "from" and its right hand side, its y input, is the source while the left hand side -  x input - is an index.  A scalar index (a simple number) operates on the leftmost dimension of the shape (zero-based), so it returns the nth element of that dimension. In a list of list of lists, it would return a list of lists.

$ threed
2 3 4             a list of list of lists
threed
0  1  2   3
4  5  6   7
8  9 10 11 returns a 3*4 list of lists.

{ can be indexed by a list, as well as a scalar. But you have to box the indexing list:
0 1 t11
A9242389TA786
JT267QK9T5AQK      unboxed, it returns row 0 and row 1

(<0 1) { t11
9                                   boxed, it returns the 1th element of the 0th row.

There are a lot of index structures - since the list of lists is pretty much the only structure, there need to be. But for now, that will do.

This 'sentence' (The J language metaphor drives me crazy, but it seems to make everybody else happy, and community rules, OK?) gives the first insight into how J handles iteration (1).
0 1 t11
A9242389TA786
JT267QK9T5AQK      unboxed, it returns row 0 and row 1.

Why? Because this is one of the iteration patterns, in effect;
FOR index_number IN (0,1)
     RETURN t11(index_number)
     END
'Boxing' the 0 1 turns off the iteration, and allows them/it to be used as a single index value. Built into the J website is a book (it's a book book as well, if you like that kind of thing) called 'cforj' (why not 'jforc' I can only put down to mysteries of dialect) by someone so modest that I haven't fathomed their name yet. A pity - I don't write C and it was still extremely helpful. This author devotes 6 scattered chapters to iteration, in increasing levels of complexity. Worth the effort. Have a recommend!

(1) J has a perfectly respectable set of imperative style 'verbs'; do, if, while &c. But I didn't start J to practise my imperative programming, so they'll have to wait.


Wednesday, July 13, 2016

J provokes self-talk (2)

OK, so I've managed to deal a deck. Now, suits and ranks for the cards. Courtesy the same source as the deal;

t12  =. <. t1 % 13) 'SHDC'

CHSSDCDSSHHSH
CDHSCDDCHSDHC
HSCCDDDSSHDSC

CDSSHCHDCCDHH

Not that it happened quite this easily. After my early success (ignoring its somewhat derivative quality) I figured I had this J under sufficient control to write something myself. I tried:

{'SHDC' <. t1%13
|domain error

...figuring that since J's OOO(1) is right to left, this would work fine. First divide (%) the matrix by 13, then round down (<.) the chuck the result into the indexing ({) thingy. As I write this I shudder at my stupidity, but at the  time I was shuddering over the extreme uselessness of the error message. WTF is a 'domain error'?

In fact, there's a lot going on here.

  1. Iteration. Something like a "for-matrix" loop is built into J. Neat. (Wait, there's more)
  2. Vocabulary. { is NOT called index. It's called 'catalogue/from'. The vocab of J is huge, and you need to have it right to navigate the documentation. I usually make up names for everything, typically translating into some amalgam of the first two or three programming languages I ever learnt, and this 'knack' has not been helpful
  3. J doesn't have helpful error messages for a beginner.
  4. a J 'verb' accepts either one or two inputs (typically - I haven't finished understanding this yet, and I dimly recall reading somewhere that you can handle three inputs) conceptually called 'x' and 'y', as in:                                                 x verb y
    • since J is right-to-left, this means that y  is the primary input. Now, quite frankly, this is whack. When I learned algebra - and I bet I'm not alone in the universe in this - x  came before y.  I did a lot of exercises with x before y ever turned up. And I don't want to be rude, but x comes before y in the alphabet. Since, in J, right comes before left, what the hell is wrong with 'verb x' ? OK, in the end it's all arbitrary, and I'm getting the hang of it now, but is the motivation for this worth the intuitive cost? Or did Roger Hui(2) start his algebraic career with y ?
    • what I now think the error means is that I wasn't providing an x for ...in other words, the output of a function only gets passed to the y input, without some advanced cleverness.
I didn't work any of this out at the time, mind you. I just stopped thinking so much, and went back to direct plagiarism.

t11  =.  (13 t1) 'AKQJT98765432'

A9242389TA786
JT267QK9T5AQK
JQ6Q964AK5J75
85J334472T3K8

I haven't bothered to highlight the parentheses, because they're not on the J vocab page. This last expression adds assignment (=., or =:) and remainder (|).

It still doesn't look much like a deck of cards. Back to Alvord & Thomson to find that they 'laminate' (,;) the suits and the ranks together.
t11 ,: t12
A9242389TA786
JT267QK9T5AQK
JQ6Q964AK5J75
85J334472T3K8

CHSSDCDSSHHSH
CDHSCDDCHSDHC
HSCCDDDSSHDSC

CDSSHCHDCCDHH

Yuk. OK, research in the vocab book gives 'append' (,) and 'stitch' (,.)

 t11 t12
A9242389TA786
JT267QK9T5AQK
JQ6Q964AK5J75
85J334472T3K8
CHSSDCDSSHHSH
CDHSCDDCHSDHC
HSCCDDDSSHDSC

CDSSHCHDCCDHH

t11 ,. t12
A9242389TA786CHSSDCDSSHHSH
JT267QK9T5AQKCDHSCDDCHSDHC
JQ6Q964AK5J75HSCCDDDSSHDSC

85J334472T3K8CDSSHCHDCCDHH

Guess Alvord and Thomson aren't card players. A lesser man would stop here and find out exactly what is going on, but it doesn't look worth the investment of time to me. I checked the vocab page for 'interleave' but didn't find it. Plan B.

Sitting on the bus I've been trying to memorise the vocab from a handy pocket cheat sheet. One of the things I have found is that you can get a row from a matrix with { . This doesn't seem to gel all that obviously with what i know about { (3), but OK. I can try.
0 { t11
A9242389TA786
0 t12

CHSSDCDSSHHSH

Good. And laminate.
 (0{t11) ,: (0{t12)
A9242389TA786

CHSSDCDSSHHSH

Do not think that will work without the parentheses, and don't think I'm going to explain why yet, either. It aligns if I 'box' (<) it.

 < (0{t11) ,: (0{t12)
┌─────────────┐
│A9242389TA786         │
│CHSSDCDSSHHSH   │

└─────────────┘but it doesn't cut and paste very well. How about
<"0 (0{t11) ,: (0{t12)
┌─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┐
│A│9 │2 │4 │2│3 │8 │9 │T│A│7 │8 │6 │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│C│H│S │S│D│C│D│S │S │H│H│S│H│

└─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘Likewise. Well, I can ignore the design for now. There's rather a startling amount going on. Plus, this is just an unsorted hand, virtually no use at all. A lot to do yet.


(1) Order of Operations
(2) A very important person in the invention of J

Sunday, July 10, 2016

J provokes self-talk (1)

I note that the last time I wrote anything about anything here, it was about programming. Amazingly, this is too. There might be something in that...

Anyway, in the intervening period I've taught myself - poorly - QWERTY touch typing. Go with the flow. I'm sure DVORAK is better, but for sure it's not so easy to have every keyboard in your life set up that way. We share computers at work.

Frink, sadly, is no longer part of my life. It's a great implementation of a language; everything you need to know on one page of extremely clearly written HTML; nothing too idiosyncratic in the syntax & a very clean implementation of classes. Someone should give Alan Eliasen (the author) a prize. But I found it was too difficult to publish Frink programs - people didn't feel comfortable, neither downloading it nor using it. So I switched to Javascript to build apps using PhoneGap / Cordova. This I succeeded in doing reasonably effectively. Yes, JS is a crappy kind of language, but I'm a crappy kind of programmer. Pragmatically, I can now develop for the internet & mobile platforms more or less easily. I can use Node for the desktop - although actually I've been using Python/Tkinter to build GUIs to support the JS application data. (That was really a terrible idea, but I stumbled across a couple of easy-to-understand websites on building GUI's in that framework, so it was easy. Now I've found 3 better GUI managers for Python than TK, but I can't be bothered. Admin GUIs are not a priority). Javascript has turned out to be precisely as productive a tool as I suspected.

Now, my life strategy is to avoid productivity by flitting from one thing to another. Right now I have a working Android app in alpha (with a long list of excellent enhancements); a working web app likewise; and a ton of (functional, useful) ideas that would leverage off the accumulated intellectual capital of those two projects.

So I've decided to teach myself J.

No-one in their right mind would do this - except that it may be unequaled as a way of wasting time. Here, then, are some of my rationalisations.


  • It's a very interesting language, as in that mythical Chinese curse. And it surely is.
  • It's a mathematician's language, and on the list of things I have to do is to improve my maths to a point where I can understand physics beyond baby talk. Well, of course, everyone should.
  • It has a very dense vocabulary, and I have always hated verbose languages. Otherwise I'd have been a Java programmer 20 years ago. Density trumps accessibility; hence literature.
  • It's a club! Who doesn't want to belong?
  • It's a very different way of thinking about programming. Maybe that will improve my day-to-day programming, in the same way that, say, a 27 handicapper plays better golf using a centre-weighted, carbon-shafted one iron off the tee, instead of Big Bertha.
  • It has an interesting history. Like LISP, Scheme and Prolog (every programming language), except more obscure.
  • Sadly, I enjoy the masochism of teaching myself difficult things. The frustration, the self-hatred, the bad language, the tremendous sense of achievement when something actually works.


So here's how the first month has gone.

I decided to write a program to produce bridge hands according to specific criteria of strength and distribution. There's a couple of programs around that do this already, two that I know of in TCL and Python, but I thought it might be fun to play around with my own. (1)

There's quite a lot of documentation on J, but frankly, it's not very a ccessible. Someone says somewhere that you should read everything three times; once to get a general idea, once to understand specific things as they come needed, once to 'bring it all together'. That's sort of true - if for 3 you read '3*  *: 3' (I think that's right, although 3^3 is more guessable). Everything seems to be documented, but finding it isn't easy and understanding it is harder. There are two main prose styles; comprehensible but not useful,  and comprehensive but not useful. Save the 2nd category for the last 3 times of reading, when it may turn out to be a little harsh.

It is there though, so you can solve every problem you have if you bash your head against it hard enough and long enough. Plus, as an added bonus, you could easily be smarter than me.

If wasn't very hard to find a sample program for dealing hands. Alvord & Thomson's Easy-J gives you this:
13 ? 52
Brilliant, hey? Select 13 numbers between 0 and 51 without replacement. All done by the handy-dandy dyad ? . Bridge has 4 hands, not one, of course, but it didn't take long to work out that:
4 13 $ 52 ? 52
does the trick. 4 13$ produces 4 13-element lists, populated by the 52 random-without-replacement bit. OK, we have:

18 48 20 45 15 3   35 32 0   7    13 22 31
16 5   28 19 8   43 27 41 17 14  40 38 44
21 4   50 29 51 10 47 24 34  6   26 30 37
39 42 1   23 2   11 49 12 25  46 36 33 9

This has taken about 30 minutes. I'm excited. J is awesome. I mean, it wouldn't take long to write the code to do that in any language, but it would certainly take up more space on the page. What next?

(1)And actually, thinking in J *has* given me an idea, which might be interesting, about handling distributions via the $ (shape) verb.
*Update - a bad idea.