Wednesday, 18 October 2017

AI and the Last Human Game

Poker has long represented a special kind of challenge to AI programmers, and not just because of the huge financial reward to be had by anyone who cracks it. Unlike games such as chess which can be 'solved' by little more than brute computational force, the face down nature of some of the cards in most forms of poker means that each player has only partial information about the state of the game/hand at any given moment and so has to make inferences about what those cards might be from clues they pick up in their opponent's behaviour. Additionally, everyone plays the game differently - some players bluff more frequently than others, or bet with marginal hands when others wouldn’t, or tends to call an opponent’s bets hoping to catch them bluffing more or less, etc. - which reflects the fact that there are many ways to play winning poker against every different opponent.  (Contrary to the picture painted by Hollywood, body language 'tells' that expose whether or not an opponent is bluffing are rare even in 'live' games and in reality clues are gathered from an understanding of an opponent's tendencies built up over time and then combined with their behaviour in the current hand.)

By its very nature then, poker is a game which is impossible to play 'perfectly' (where 'perfect' is defined as the way one would play if you were able to see your opponent's face down 'hole cards' and then adopted a style that optimised your winnings given that information). The aim is always to play in a way that is as close to perfect as one's temperament and analytical skills permit, with a great deal of emphasis placed on adapting one's strategy to best exploit the nature/tendencies of a given opponent.

The phrase ‘risk intelligence’ has been used to describe the kind of thinking crucial to playing good poker. It implies an ability to assess probabilities in a context of limited information, a deficit which is compounded by the fact that much of the information one does have comes via the medium of idiosyncratic and inconsistent human behaviour, and then formulate an exploitative strategy that anticipates future changes in that behaviour in the face of dynamically unfolding events. Put another way, it requires a mix of cognitive empathy and adaptive Game Theorising that engages fully with the intricate messiness that is each unique human mind. No disrespect to Gary Kasparov, but chess is child’s play in comparison.

The kind of flexible, imperfectly informed, real time risk/reward thinking that high level poker demands was seen by many, myself included, as quintessentially human, and, as games like checkers, backgammon and even Go succumbed one by one to challenges from AI, I started to ruefully call poker The Last Human Game.

Even a decade or more ago, talk was always rife in the poker scene about the day when the online game would be destroyed by 'bots' who would play with the optimum combination of aggression and deception, and run rigorously randomised rings around even the best human players. But as is often the case in AI research, that seemingly impending day never quite came. True, Limit Holdem, a far simpler game with fewer variables to consider due to the proscribed bet sizing rules, fell to the machines some years ago. But No Limit Holdem (in which a player can bet any amount they like, henceforth referred to as NLH), especially in its ‘heads up’ (that is, one-on-one) form, seemed an insurmountable challenge. (Somewhat counter-intuitively, it's generally felt that the complexity of the game increases as the number of players at the table falls, with the scope for creative and adaptive play far greater in a ‘heads up’ encounter than when facing a ‘full ring’ of opponents, where a more formulaic approach can be adopted. Thus, heads up NLH has always been seen as the ultimate testing ground for Man vs Machine.)

As recently as 2015, a purpose built AI designed by a team from Carnegie Mellon Institute lost to a team of poker pros playing heads up NLH, in conditions that controlled for variance (or 'luck') by, amongst other things, having each hand dealt twice, the second an inversion of the first, so that both sides got to benefit equally from the 'run of the cards'. This ‘variance control’ also allowed the teams to examine hands from both sides after each session was played, giving them an opportunity to learn how the other was playing and adapt their strategy accordingly.

Although the humans won that battle (an epic three week slog in which the four pros each played in isolation for 10hrs a day), if you spoke to any of them after the match one could sense they felt the writing was on the wall. And so it has proved. Last month the same Carnegie Mellon team came back with Libratus, their improved poker AI, and faced four of the best NLH heads up pros in the game.

It wasn't even close.

Crucial to Libratus's success was its ability to play unpredictably and to learn quickly as it went along. Initially, the human team felt they had identified some significant weaknesses or 'leaks' in the AI's game, and hammered on them relentlessly. But as night fell and the humans slept, the AI was learning - crunching the data from the day’s play, running simulations, refining its strategy. Each day it came back with yesterday's leaks plugged. Soon enough, the pros stopped being able to find any significant holes in Libratus's game at all.

Libratus, however, had found plenty of weak spots in the humans' game. On top of the obvious advantages of never fatiguing or losing its cool, it also played in a style that was as creative and unusual as it was brutally efficient. Rather than adopting a conservative or 'tight' strategy as previous generations of poker AIs have done (and which would work well against weaker, more predictable, less adaptive human opposition), this AI liked to bluff, and bluff big. Often betting far more than the size of the pot - that is, risking far more than it could possibly win in order to make its opponent fold the best hand - it stole hundreds of pots that a human player, faced with the same situation, would usually have given up on. Crucially, it did so with such a well-balanced range of hands - sometimes as a bluff, sometimes with a very strong holding - that the humans couldn't counter it effectively.

Two thirds of the way into the match the AI had built an insurmountable lead and was only getting stronger. Over 80,000 hands it was up $750,000 on the humans. Luckily for Team Humanity, this was only a notional loss with no real money being waged. But as one of the human players, Jason Les, quipped, "It’s not about the money, it’s about preserving human dignity. And it’s not going well.”

Looking ahead at a world of dwindling human exceptionalism, I found myself wondering what areas might yet be preserved for people alone. Les's gallows humour in the face of certain defeat seems like as safe a bet as any. After all, the need for comedic consolation in the face of impending annihilation probably won't be within our omnipotent robot overlords' direct range of experience. But then, who knows? After simulating a few billion iterations of possible existential threats on their way to total domination, they might have stumbled on a choice gag or two. And if we're really lucky, they could even use that humour to temper the brutal efficiency of their rule.

Tuesday, 10 November 2015

American dynamism vs French sclerosis: More cheese-eating, less cheese-making?

Warning: this post contains graphs. Many wonderful graphs.

According to conventional (read: conservative) wisdom, the Left has a problem. Sitting across The Channel, stinking of body odour and brie, lie the French. Literally lying there - lounging about in a feckless stupor of institutionally enforced laziness and cultural arrogance, surrounded by the decaying glories of their past, housed in a nation-state cum mausoleum along with their naive dreams of socialist economic rationality. Corbynites take note.

As if that wasn't bad enough, across The Pond in swaggeringly individualistic America reside a people who produce a huge proportion of the World's goods and services, innovate like maniacs, and, with the American Dream by their side, appear to actively relish their government's light-touch regulation and minimal social safety net. A promised land of laissez-faire creative destruction where hard work and a limber mind will get you far, unhindered by the deadening hand of the state.

So what's wrong with this picture? I mean, apart from the hyperbolic rhetoric and casual racism. OK yes, the French enjoy more free time and more high quality public goods. They even live a bit longer, too. Oh, and they have more celebrity philosophers. And yes, the Americans aren't all so enamoured with capitalism in tooth and claw. Despite being significantly richer on a per capita basis, they also have to endure higher levels of poverty, and lower levels of public health and education, along with many other human development indicators. Plus, ya know, Kim Kardashian and Donald Trump. But with an economy so much more productive and healthy than calcifying, atrophying France, even these problems will eventually fade with the panacea of boundless prosperity while across the ocean, the French drown in a sea of red tape and debt. And fondue. A debt-fueled death spiral into a lake of liquid cheese.

But surprise! The main problem with that picture is that it's total bollocks. Or couilles totales as the Lazy Ones probably don't say. 

More accurately - and as with most myths - it's a fabrication based on a few fragments of actual fact, supplemented by a few more 'facts' taken out of context or which used to be true but no more, and then just piled over with mountains of cultural stereotyping (you know, like in 2002 when Dubbya said, "The problem with the French is that they don't have a word for entrepreneur"). 

More on the fun stuff in a bit. But first, here are some of those real facts:

 - There's no denying it, in terms of total GDP growth things look pretty ugly for France:

Figure 1
- Average GDP per capita in the US really is more than 25% greater than that of France (2013: $53,041 vs $42,503).

- The relative size of the state compared to whole economy is vastly different, and the gap's widening too:

Figure 2

 - The unemployment rate seems to confirm the notion that the French are either lazy bastards, or simply can't create enough jobs for themselves. Or both. (And the momentary convergence during and after the 2008 crisis only shows how inflexible and coddling their labour market is, too - bloody cry-babies.):

Figure 3

 - Youth unemployment looks even worse:

Figure 4

Pretty hard to come back from that to glorious victory you'd think. And you'd be right: I'm not about to pull a statistical rabbit out the hat here. France isn't somehow going to outstrip the US in a range of indicators that render the above totally meaningless. But if you peer a bit closer and add a little context, suddenly the French don't look like total basketcases either. In fact, these days there's not a huge amount to pick between them and their thrustingly entrepreneurial American cousins. 

Take total historical GDP growth. That huge disparity in Figure 1 is almost entirely explained by US population growth as this GDP per capita graph shows:

Figure 5

And you only have to peel back the surface layer for the GDP per capita to look quite different, too. For example, between 1975 and 2006 the US average GDP per capita grew by 32.2%, whereas France's lagged with only 27.1%. However, if we exclude the top percentile from the data we can get a far better idea of what's happening to the remaining 99%. In this case the French GDP per capita grew by roughly the same amount, 26.4%, whereas for the US the figure drops off a cliff down to a mere 17.9%. The Occupy movement's rhetoric about the massive enrichment of the 1% at the expense of the 99% in the US turns out to be depressingly true. Indeed, the bottom 50% in the US have seen real incomes (incomes adjusted for inflation) flatline or even fall for the last 40 years. 

What about unemployment? And especially youth unemployment? If one focuses on 'prime age' employment - that is the employment rate of those aged 25-54 - amazingly France out-performs the US:

Figure 6

What then explains the high youth and older age unemployment in France? Well, the latter is no mystery: they're all retired living on generous pensions playing boule, knocking back claret and writing obscurantist existential philosophy with which to torture the undergrads at the Sorbonne. But the youth picture seems more troubling, and it's true that the high minimum wage and other structural factors probably do account for some of the difference. But it's equally true that rock bottom tuition fees and generous in-education benefits mean that French students aren't forced to get jobs to pay for their university career in the way that many Americans are. 

At this point I think it's reasonable to ask, what do we want from a national economy? If the answer is that we want greater power and glory for our nation then the US wins hands down. But if the goal is to provide the sustainable economic conditions to enable our population to live materially and socially fulfilling lives, well, I don't wanna say that France necessarily triumphs in this head-to-head, but in economic terms it's a pretty close call. 

But what about that French (un)sustainability? Mountains of public borrowing and debt leading to inflation and sky high interest rates and then a loss of international competitiveness? Not exactly, no:

Debt levels aren't too different:
Figure 7

No sign of soaring prices:

Figure 8

The French actually pay less to borrow than the US:

Figure 9

And their trade balance, although not perfect, is actually marginally better than that of the US:
Figure 10

OK OK, enough graphs. So what's the takeaway from all this? Well, the main one is that there's more than one way to skin an economic cat. The US and France have long adopted divergent economic policies and yet on the majority of economic measures people care about, they appear to have converged anyway. That's not to say that policy doesn't matter at all, of course it does. Internally, the degree of inequality is clearly impacted by a government's fiscal strategy. Also, there are limits to how irresponsible in either direction a government can be and still rely on the underlying strength and resourcefulness of their people to see them through. But within what appear to be relatively wide bounds, people and economies just seem to get on with it, leading to outcomes that differ far less than we are led to believe.

So then why the apocalyptic rhetoric in the anglo-saxon world about the horrors of turning into everyone's favourite Gallic whipping-boy? It can only be politics. The French have long sought to balance the books by raising taxes on the rich. And in an act of rare global solidarity, the rich - who it must be noted, generally control both the government and press - have united in their defence of their shared values. That is, their wealth. Oh, and freedom! Mustn't forget freedom. The freedom of every man, should the fancy take him, to pull on a one-piece and take a nice leisurely dip in his pool.

Monday, 30 March 2015

UKIP and psychology of racism.

That UKIP voters dislike immigration is a banal truism. That UKIP voters have little personal experience of immigration, however, is more surprising. A recent DEMOS report showed a strong correlation between relatively white homogenous areas and burgeoning UKIP support. A weaker, though still significant, correlation was also shown between areas of low unemployment and high UKIP support illustrating that the 'they took our jobs' theory of electoral popularity some (not least the supporters themselves) have identified is also less than convincing.

An averagely thoughtful liberal cosmopolitan could draw the fairly simple conclusion that, whereas living in a diverse community like London tends to make one more open minded to sharing public space with the capital's rainbow of immigrants, the insular life of homogenous South Thanet (96% white British) serves only to reinforce the sense of Us and Them and so heighten intolerance of immigrant outsiders. However, if one was being less charitable, as those on the left who view Farage and his clan as nothing short of satanic seem to be, one could simply accuse UKIPers of gross bigotry and racism, chuck a few eggs at Farage's face and feel thoroughly pleased with oneself for a job well done.

Now, I readily concede that the UKIP leader does have one of those faces that cries out for corporal punishment. And, even on their own terms, the party's policies are often deplorable when not merely incoherent. But seeing the bile being vented on social media by self-righteous 'activists' whose own smug visages seem no less deserving of an encounter with a blunt object, perhaps a little moderate corrective is in order.

I happen to be reading the developmental psychologist Paul Bloom's book, Just Babies: The Origins of Good and Evil, and found his chapter on 'Others' - as in, evil immigrants, filthy foreigners, etc - instructive. The seeds of racism, Bloom argues, run deep and start to sprout early, but the picture isn't as simple, or as bleak, as it first appears.  To start with, babies probably aren't natural born racists at all; they simply prefer the familiar, regardless of race. But that's not the only reason to doubt the veracity of the notion that racism a naturally occurring inevitability.

Evidence suggests that as adults we tend to encode three pieces of information about a new person: age, sex, and race. Intuitively this makes sense. But, as psychologists Kurzban, Tooby, and Cosmides pointed out in an influential paper, if one considers the circumstances under which these kinds of psychological habits formed, the race component of that triad starts to look a bit odd. After all, our ancestors would have had no contact with anyone belonging to what we call a 'different race' so why do we fixate on it now? As Bloom tells it, "race matters only insofar as it piggybacks on coalition".

Coalition refers to the simple practice of breaking the world into Us and Them by whatever means available, something that would have been invaluable in the violent localised inter-community conflicts that characterised early humanity. To test their hypothesis, Kurzban et al used a method known as the memory-confusion paradigm, which Bloom describes as follows:

"They gave people a series of pictures of people's faces, each with a sentence attributed to that person. Later, they asked participants to recall who said what. Given enough picture/sentence pairs, participants inevitably make mistakes, and those mistakes reveal what characteristics we naturally encode as meaningful. If one hears something from a young Asian woman and later forgets the source, it is more likely we'll attribute it to a young Asian woman (or another young person, or woman, or Asian) than to an elderly Hispanic man."

But Kurzban's study contained a twist. They used pictures and statements from an equal number of black and white people, but then randomly assigned them distinctively coloured jumpers. When they did this, the mistakes participants made were far more likely to be based on jumper colour than race.

Further evidence for the non-universality of racism can be found in research with children by McGlothlin and Killen. They designed a study using images of ambiguous moral situations containing an ethnically mixed cast and asked white children between the age of six and nine to describe what they saw. In some cases, the children displayed racial prejudice in how they analysed and recounted the scene, but only if they'd been to all-white schools. This supports what's known as the 'contact hypothesis' - the notion that, under the right circumstances, social contact diminishes prejudice.

So, the evidence clearly shows that coalition forming and group loyalty are powerful social forces written into our DNA which can lead to intolerance and racism. But they don't have to. In this light, the UKIP voters so demonised by the left are largely victims of circumstance. They fear immigrants for their 'otherness', and for how they'll change their communities, but don't realise that through the process of exposure they'll almost certainly change their own attitudes regarding the Dreaded Other.

As for the protesters who spoiled poor Nigel and his family's pub lunch the other day, someone needs to tell them that their one-off flamboyant efforts were wasted. What they really need to do is shut up and move in next-door.

Wednesday, 17 December 2014

Response to 'An open letter to Russell Brand'

Russell Brand staged a protest/publicity stunt at RBS. An employee called Jo who briefly interacted with Brand outside, and who seems to have been annoyed that his lunch got cold, wrote him an open letter as a result. It's been getting a lot of attention, especially in the right-wing press. This is my dashed off response to that letter.

In essense, I found the post deeply flawed and annoying when not merely tedious. It's just Brand's own rhetorical tactics - faux-familiarity and knowing faux-sympathy - made less earnest and more condescending without any of the much needed upgrade in quality control or practical empathy. Even more so than Brand's efforts, it seems to be pure choir preaching and, by its swaggering and obnoxious lack of self-doubt, it turns off someone in the middle, like me. For example, the blog's subheading, 'The worst thing about being cynical is being right', makes me want to actively assist this auto-fellatio enthusiast with a hand on the back of his head, if only to hasten the moment when he publicly chokes on his own cock. 

I mean, I think Brand is as much of a misguided simpleton as the next guy, except if the next guy is the author of this blog post who seems intent on giving Brand-mockery a bad name.

Leaving aside the completely unnecessary ad hominems directed at his messy past, it seems there are only two substantive points: 1) Whether or not his publicity stunt has a role to play in advancing the scope and depth of our public discourse (he spends a couple of yawn-inducing preliminary paragraphs 'showing' that a publicity stunt is all it was, as if that needed pointing out, but whatever - maybe he feels that his readership require baby steps... actually, judging from the parade of gormless comments beneath it, perhaps they do). And 2) The defence of the bank bailouts as being in the public interest rather than symptomatic of something deeply wrong with the system. 

There are also some accusations of hypocrisy sprinkled about, laced with crude attacks on the Beeb, which are entirely irrelevant, but if anyone thinks they're valid I'll treat them in a later post.

So, 1) Does the publicity stunt have merit? In the context of a broader strategy of public engagement, someone in marketing would certainly say yes. Any publicity is good publicity, so people talking about the topic you're interested in - as we are doing here and as the blogger has ironically encouraged with his post - increases public awareness and engagement, albeit somewhat superficially in the main. From the perspective of someone who supports Brand's position, as long as some of these new engagements lead to an onging and deeper engagement in the issue for a few individuals, this is clearly net positive. It also maintains momentum, which is crucial in social and political campaigning. So, strategically, this is a no-brainer. 

2) No mention of moral hazard, no mention of too-big-to-fail, no mention of implicit taxpayer subsidy of risk leading to increased profitability, no mention of regulatory capture. I could go on. If this guy actually thinks that the RBS bailout is net positive, even in purely financial terms, for the average citizen, then he's lost in a pit of ideological spin and self-affirmation that is likely terminal. As far as I'm concerned the matter is so clear cut it's barely worth discussion. (I say this as someone who supported the bank bailouts as being the least bad option in the circumstances. The point is, there was nothing inevitable about the circumstances. On this point, Brand and I are in agreement.) 


Ultimately, attacking Brand personally for his misguided and simplistic politics is a bit like attacking a hipster for being a fashion victim. It's shooting fish in a barrel and only serves to make the shooter feel better about themselves. It's pathetic. If you want to engage with the Brand/Farage phenomenon, have a crack at the populist political soup of dysfunction and apathy he/they bubbled up from. That's where the real questions lie. But the answers are harder divine, and generally more discomforting, too. Much easier to rattle off a flabby tirade at some easy target on the other side. And it makes your peers think you're cool. Just ask Farage and Brand: it's exactly how they've gained popular support themselves.

Monday, 19 March 2012

Efficient packaging from Fresh Direct

I recently ordered a 'bunch' of groceries for delivery from Fresh Direct, the New York equivalent of Ocado in London. Over packaging is the norm here as everyone knows but this is a high end grocer with an emphasis on fresh, organic goods (the goods arrive in recycled boxes no less!) so I dared to think that they might have a sane packaging policy as well. Sadly no. That one extra plum tomato apparently just can't quite fit in with the other 5. Dito for the two lonely carrots sitting next to their more sociable brethren.

Wednesday, 14 March 2012

Musings on mediocre air travel

Delta. To my knowledge not an airline of great renown, as evidenced by the fact that less than one in four seats on my incredibly cheap transatlantic flight are occupied. They have made an effort with the uniforms though; the entire crew dressed in black polo necks.

This swish retro look, that according to the Daily Mail 'never went out of fashion!' and according to the Guardian did go out of fashion but is now making a comeback (heaven help us), was somewhat offset by the largest and campest of our stewards who breezily reported 'sweating like a school boy in church' as he served me lunch. I liked his customer friendly version of the classic 'whore in church' phrase, substituting, as it does, the reference to female sexual and economic exploitation for an image of teenage guilt ridden masturbation. My appreciation clearly didn't go unnoticed as he quickly returned to surreptitiously top up my virgin mary with a free shot of vodka before smiling peculiarly and saying, "see! Treat me nice and I treat you nice". As he turned and presented his vast arse to my face another somewhat unpleasant image was thrust into my minds eye and my head started to spin. Just how nice could he treat me? And would it involve that great gluteal protuberance in any way?

I feel uncomfortably certain that New York has many more such thought provoking encounters in store for me.

Friday, 9 March 2012

Thomas Paine Quotes

Time makes more converts than reason - from the opening paragraph of Common Sense