The Software Architects' Newsletter December 2019

A monthly overview of things you need to know as an architect or aspiring architect. [InfoQ]( The Software Architects' Newsletter December 2019 [View in browser]( In our twenty-ninth issue of the Architects’ Newsletter we are taking a break from our regularly scheduled program to bring you a 2019 “year in review” edition of the newsletter. In the list of highlights below we have chosen a combination of our favourite articles, and also the most popular articles with readers, as well as some new content items that build on the themes we’ve covered over the last year. News Highlights for 2019 Weave Ignite Builds on AWS’s “Firecracker” microVM for Container and Serverless Workloads At the end of 2018, Amazon announced the [release of Firecracker](, an open source “micro” virtualization technology that is purpose-built for "creating and managing secure, multi-tenant containers and functions-based services". In 2019 there has been lots of innovation around this technology, most notably by the Weaveworks team with the creation of Weave Ignite, which combines Firecracker MicroVMs with Docker/OCI images to unify containers and VMs. Weave Ignite also supports the increasingly popular [GitOps]( style of deployment, and can manage VMs declaratively and automatically like Kubernetes and HashiCorp’s [Terraform](. Bryan Liles on Making Kubernetes Easier for Developers In this [InfoQ podcast, Daniel Bryant sat down with Bryan Liles](href>, senior staff engineer at VMware. Liles mused that cloud native platforms like Kubernetes are complicated, as there are lots of moving parts. Although Kubernetes is fast becoming a cross-cloud fabric, the most important challenge to be tackled to increase the adoption of frameworks like Kubernetes is “how do we move code from our IDEs to wherever it needs to run with the least amount of friction?” Lightbend Released CloudState, Providing a Spec for Managing “Serverless” State and a Reference Implementation with Knative Lightbend has released [CloudState](, an open-source serverless framework designed to implement stateful management on the Knative and Kubernetes stack. Lightbend, the team behind Akka and the Play framework, have stated that CloudState consists of two components: a standardization effort to define a state management specification, which includes a protocol and a Technology Compatibility Kit (TCK); and a reference implementation, which implements the backend and a set of client API libraries in many languages. An Engineer’s Guide to a Good Night’s Sleep [Speaking]( to Shane Hastie, The Financial Times’ Sarah Wells noted, “It is a big change for most developers to be operating software. And if you haven’t done it before, it is terrifying the first time you are called up at 2 in the morning and you have to fix something. It is up to you as an organization to turn that into something where people feel supported. Mostly the way we do that is we have tended to swarm on problems.” In “An Engineer’s Guide to a Good Night’s Sleep,” [Nicky Wrightson](, principal engineer at Skyscanner, explores the same topic and asserts that leaders should build teams that [understand what being on call means](. If a team owns the system, they also own all the aspects of supporting it. Event Streams and Workflow Engines: Apache Kafka and Zeebe Apache Kafka is a highly scalable, distributed, streaming platform often used to distribute messages or events within a microservices-based system. These events are often part of a business process, with tasks spread over several microservices. To handle complex business processes a [workflow engine]( can be used, but to match Kafka, it must meet the same scalability Kafka provides. [Zeebe]( is a workflow engine currently being developed and designed to meet these scalability requirements. In a joint meeting in Amsterdam, [Kai Waehner]( described features of Kafka and how it fits in an [Event-Driven Architecture (EDA)](, and [Bernd Rücker]( described workflow engines, Zeebe, and how it can be used with Kafka. In addition, Rücker also joined InfoQ for a podcast focusing on “[architecting for scale](”. MicroProfile Releases Reactive Streams Operators 1.0 The Java-based MicroProfile community have released the [Reactive Streams Operators 1.0 API](, a specification that defines a set of operators for [Reactive Streams](. Inspired by the [java.util.stream]( package, first introduced in JDK 8, and based on the Reactive Streams initiative, the Reactive Streams Operators API allows developers to create Reactive Streams, process the data transiting in the streams, and accumulate results. The Reactive Streams initiative was based on the [Reactive Manifesto](, which was started in 2013 as a collaboration among Lightbend, Netflix and Pivotal. The [initial specification](, released in 2015, was ultimately adopted with the release of JDK 9 as the [java.util.concurrent.Flow]( API. The Whys and Hows of Database Streaming In another QCon SF presentation recording, Joy Gao talks about [how database streaming is essential]( to WePay's infrastructure and the many functions that database streaming serves. She provides information on how the database streaming infrastructure was created and managed so that others can leverage their work to develop their own database streaming solutions. She also reviews challenges faced with streaming peer-to-peer distributed databases. How Design Systems Support Team Communication and Collaboration In an InfoQ news piece Ben Linders covered a talk by Stefan Ivanov, a senior UX architect at Infragistics, who spoke about the purpose of [design systems](, and shared experiences from creating [Indigo.Design]( at [ACE! Conference 2019](. Ivanov argued that by using design systems, design teams can improve their workflow, reuse their knowledge, and ensure better consistency. They allow teams to fail faster and speed up the iteration cycle, which allows them to spend more time collecting user feedback in the early stages of product design, and reach the sweet spot of a product market fit much faster. Ignite the Fire - How Managers Can Spark New Leaders In the recording of this top-rated QCon NY talk, Nick Caldwell, Chief Product Officer at Looker (recently acquired by Google), discusses the [three ingredients needed for inspiring non-manager leaders]( to emerge, and provides simple techniques that any team member can apply. Caldwell uses stories based on his experience to help illustrate how to inspire leadership at many levels in the organization, and e shares real-world advice for managers, individual contributors, and architects that want to inspire leadership and “ignite the fire”. Empathy is a Technical Skill [Andrea Goulet]( maintains that [empathy is more than a feeling](; it’s a skill that can be learned and applied. Lack of empathy while coding can lead to the creation of legacy systems that are difficult to work with. Leaving communication artifacts throughout a codebase can help other people understand ideas, rationale, and constraints. Empathy can help an engineer identify human bias and assumptions that get built into software systems. InfoQ Retrospective and Software Trends for 2019 The InfoQ team has put together [an overview of highlights from InfoQ in 2019](, important trends in 2019/ 2020, a list of conferences you should attend, and a spotlight on next year from our editors’ point of view. InfoQ’s Favourite Case Study Grady Booch: AI today is a System Engineering Problem with AI Components. InfoQ editor Michael Stiefel has written a summary of [Grady Booch’s QCon SF keynote](. According to [Booch](, most current AI systems are about pattern matching of signals at the edge and [inductive reasoning](, not true artificial intelligence. During his second day keynote at the 2018 QCon San Francisco, "[Building the Enchanted Land](", he explained his view that AI today is a "system engineering problem with AI components." True AI uses decision making and [abductive reasoning](, which allows those systems to reason and learn. Current artificial intelligence applications are far from being able to accomplish that; they are just components in larger systems. Contemporary AI is not of recent origin, indeed many of the architectures and algorithms are decades old. The difference today is the abundance of computational power, and the existence of large bodies of tagged data. In fact, working with the data to understand it, and get it into the proper format will most often take more work than the building of the model. Developing AI systems requires the same skills as classical software development: crisp abstractions, separation of concerns, balanced distribution of responsibilities, and an attempt to keep things simple. You want to grow your system through incremental and iterative releases. Of course there are unresolved problems. One of them is how to bring together symbolic, connectionist, and quantum models of computation. Booch strongly disagrees with the views of the DeepMind community that thinks that AI and neural networks are going to be at the center of systems. He called attention to [Moravec's paradox](. There are far more neurons in the human brain devoted to signal processing, the visual cortex, and the hearing cortex than there are in the decision making system. This kind of split appears in the hybrid systems of today. Artificial intelligence for the edge, symbolic systems for decision processing, and conventional software around it. As Alan Newell observed during one of the [AI winters](, "Computer technology offers the possibility of incorporating intelligent behavior in all the nooks and crannies of the world. With it, we can build an enchanted land." To Booch, software is the invisible language that whispers stories of possibility to the hardware. Today, software developers are the storytellers, the ones who are going to build the enchanted land. This excerpt is based on a full article can be found on InfoQ “[Grady Booch on the Future of AI.](” You can also hear an interview with Booch discussing AI in [this episode]( of the InfoQ podcast. To get notifications when InfoQ publishes content on these topics follow “[Cloud](href>,” “[Cloud Computing](href>,” and “[Cloud Architecture](href>” on InfoQ. Missed a newsletter? You can find all of the [previous issues]( on InfoQ. InfoQ strives to facilitate the spread of knowledge and innovation within this space, and in this newsletter we aim to curate and summarise key learnings from news items, articles and presentations created by industry peers, both on InfoQ and across the web. We aim to keep readers informed and educated about emerging trends, peer-validated early adoption of technologies, and architectural best practices, and are always keen to receive feedback from our readers. We hope you find it useful, but if not you can unsubscribe using the link below. [Unsubscribe]( Forwarded email? Subscribe and get your own copy. [Subscribe]( Follow InfoQ.com on [Twitter]( [Facebook]( [LinkedIn]( [Youtube]( You have received this email because you subscribed to "The Architects' Newsletter". To stop receiving the Architects' Newsletter, please click the following link: [Unsubscribe]( - - - C4Media Inc. (InfoQ.com), 2275 Lake Shore Boulevard West, Suite #325, Toronto, Ontario, Canada, M8V 3Y3