Myki, top-up speed, and distributed data systems
The release, last week, of both the ombudsman's "Investigation into public transport fare evasion enforcement" and the government response shed some important light on public transport ticketing and the difficulty of both enforcement and compliance. These are long-standing issues, with a culture of intimidation and fear that dates back a decade and more. Moreover, the general problem of fare-collection will only get worse. Policing a system with largely open-access and micro-payments is very labor-intensive and therefore increasingly costly. The "automated" ticketing system introduced with Metcard (actually, the short-lived scratch-tickets) fixed only the selling element of this problem. And in the case of Myki, did that badly as well.
The general problem will continue to exist regardless of the ticket solution. It is predominantly a problem of open-access and there is little chance of that changing on the train tram network, and none at all on trams. But the issue of selling tickets, and the problems Myki faces with updating passengers on their correct balance, or taking online payments can be fixed. It was telling that the government list of measures began with this change:
Reducing the time it takes to top up online from 24 hours to 90 minutes
From a user perspective it is of some but limited value. If a user recognises that their Myki balance is in the negative, then they can top-up in time for the next trip. Commonly though, a user will board a tram not knowing that their ticket has a negative balance: the lack of touch-off for users of only trams means Myki reports the previous balance, not the new one on touch-on. Myki's inability to allow users to immediately top-up their balance runs counter to the experience people have with payment systems in every other part of their life. There is a reason, but the reason relates to the technology Myki was built on. Technology that is now two decades old, built for a different time. One that has now passed.
Web Businesses Will Live in the Cloud - Marc Andreessen
Contactless smartcard systems began to be introduced in 1991, slowly gaining popularity, notably after Seoul and Hong Kong which introduced them in the late 1990s. When Myki came into being, these were the gold standard by which all ticketing systems were expected to meet. Troublingly, despite there being no market imperative to have a ticketing system of this sort, the Transport Ticketing Authority chief executive Bernie Carolan indicated in 2012 that this was also the main reason: "It was known that other transport systems were heading towards smartcard systems, so the thought was, the sooner we do it, the better."
The momentum behind RFID technology for ticketing systems shows no signs of abating, as many more have been introduced in the decade since the Myki project was started. Yet this technology is nearly unique to transport (and other government) authorities. Businesses, handling trillions of online payments, and producing billions of tickets for everything from major concerts and sports events to the self-created and managed sales of companies like EventBrite, and even transport companies like Melbourne's SkyBus use bar-code technology.
The reason related back to Andreessen's prediction, made in 1999 and quickly proven correct. In internet terms, the late-90s are a different world. Storage was expensive - the textbook for data management I used in 1998 was titled "Managing Gigabytes" - and the internet was confined to fixed networks, and dial-up. In a transport scenario for a large city like Melbourne, with half a billion trips per year, storing and accessing a terabyte of yearly travel information, and transmitting user information to roaming vehicles in real-time was unthinkable. Hence the need for distributed data storage in the form of RFID cards. The network is built into the system itself: the card stores the data - the account value and travel information - and is updated by the machines. While the 1990s computer technology Andreessen predicted would move from distributed programs (like Office) to the internet (like Google Docs) has slowly switched, transport technology remains distributed across millions of cards, and the system must maintain thousands of complex readers to reflect that distribution.
Which brings us back to online top-ups. In our modern world, being able to transfer money to an entity, regardless of your geographical position, and seeing an immediate credit is standard (and indeed has been for ten years) The inability of Myki to register a top-up immediately is incongruous. However, while Myki was built in the internet era (though not the wireless one), the underlying technology was not. A monetary transfer to a card needs to be sent to the card reader in order to update the card. Moreover, because there is no way to predict which card reader a user will use, it must be broadcast to all card readers. The data involved is not large - not least because few people use online top-ups when they won't register - but it is clumsy and difficult to manage, and results in other perverse effects.
Most of the "problems" of Myki are built into the distributed system design and the need for (expensive) chipped cards:
- The most obvious of these is the disconnect between an account and the card. Because the data is on the card, losing it means losing the data, with a complex replacement process of loading old credit onto a new card and re-linking it to the account. If there was no account, the credit is lost forever.
- Single-use tickets are problematic when cards cost several dollars to create, and aren't just a signifier of trip eligibility as they were under Metcard and earlier ticket systems.
- Expiring cards - presumably a measure of the lifespan of the chip itself - is a product of the card being more complex and therefore needing to be periodically replaced.
- The complexity of distributed storage that requires software checks for account changes, partial card updates, and uncompleted trips; and expensive hardware/software roll-outs to the readers when policy changes are made
Distributed systems are rarely used in modern web programming in favour of central storage because data and network connections are now cheap and reliable. In a centralised system the data the user has to have is much simpler: either an account identifier that allows them to be recognised; or a trip identifier, with the relevant date and access encoded.
The problems identified above with Myki would disappear, as a consequence:
- Online top-ups will reference the central account, which is the same database an inspector will check. A user could receive a low account warning when boarding the tram, and use a mobile app to add credit to their account immediately without even needing to re-touch-on - the current trip counting as pending.
- Because a "card" is only a bar-code or equivalent identifier, they are easily reprinted (or regenerated if security is a concern), could be stored on a phone display, rather than a paper or plastic version, and extended to multiple users without any interaction with PTV.
- As already discussed, single-use tickets are paper versions of temporary accounts - or alternatively, real accounts that can be topped up, but printed on paper card, without the expense of an RFID chip
- Account expiry would not need to occur, though unused accounts might be archived for data reasons, and the lifespan of the card is irrelevant if a user can reprint on demand - or never prints it at all.
- And finally, centralised software means that all users (and machines) are always working with the most current version. It vastly simplifies the hardware requirements, as the only requirement of a bar-code style system is a display (an embedded browser is sufficient) and a bar-code reader that triggers a network call. There is no possibility of a partial update (as there often will be if a transaction is attempted but the card removed prematurely), as the transaction is operated at the network level. The only concern is unreliable networks - which might be a problem on some trams or buses - but it is a problem that is rapidly disappearing, and one that can be solved with cheap hardware investments.
With the contract for Myki coming up for re-tendering, the government is caught between two unpalatable decisions: continue using a system that is widely disliked, works badly (even by the standard of contactless smartcards) and by design includes incongruous limitations that are not easily fixed (if at all); or they can invest in a system that reflects modern programming and technology, but equally worryingly, be at the bleeding edge of transport ticketing systems (if still behind the curve in industry terms). The upsides to the latter approach include the possibility of a vastly superior system that is much cheaper, and the opportunity to on-sell it to other cities if successfully implemented. The downside is the possibility of another multi-billion dollar IT ticketing disaster to follow one from a decade ago. A business with Myki would be haemorrhaging customers and need to invest in better systems. A monopoly provider of ticket services can afford to be terrible. Its failures can be passed on to customers as fines for non-compliance, or the government in lost revenue; its added maintenance costs are borne by tax-payers. There is no easy choice, but it would be rare to find a user of Myki who wouldn't prefer something vastly superior.
4th June, 2016 20:52:20
Myki, top-up speed, and distributed data systems
Interesting read, so thanks. I can't agree with your last sentence. My Myki is auto topped up and just works and I don't have a problem with it. I expect here are many like me. The only issue I have is slow readers on trams, buses and some stations and while I realise the impact of slow readers at busy stations, it is not really a big issue on trams and buses, or most stations.
Andrew 5th June, 2016 20:53:06