InfoQ Special Report - Key Takeaways and Learnings from QCon London 2022 and QCon Plus May 2022

Use Code INFOQCLOUDIL22 to register for FREE. [InfoQ Live Logo] Key Takeaway Points and Lessons Learned from QCon London & Plus 2022 QCon [international software development conference]( [returned to London]( this past April and online with [QCon Plus]( in May. The conference explored the major trends in modern software development across 15 individually curated tracks. The 2022 QCon London and QCon Plus tracks featured in-depth technical talks from senior software practitioners covering [developer enablement](, resilient architectures, modern Java, Machine Learning, WebAssembley, modern data pipelines, the emerging [Staff-Plus engineer path](, and more. InfoQ Editors attended both conferences. You can read their [coverage online](. This Special Report summarizes the key takeaways from five talks presented at QCon London 2022 and QCon Plus May 2022. SPONSORED Cockroach Labs [Cockroach Labs]( [Architecting for Scale [Two Free Chapters] - Download Now (By O'Reilly)]( LaunchDarkly [LaunchDarkly]( [Effective Feature Management - Download the Book (By O'Reilly)]( Architectures You’ve Always Wondered About Shopify’s Architecture to Handle the World’s Biggest Flash Sales by [Bart de Water](, Manager @Shopify Payments Team Bart de Water talked about Shopify’s architecture to handle what he called the “world’s biggest flash sales.” His presentation was very technical, covering details about the backend built on Ruby on Rails with MySQL, Redis, and Memcached, with Go and Lua in several places for performance reasons. The frontend is done in React and GraphQL, with the mobile application written in React Native. He focused mostly on the storefront and checkout because these have the most traffic, the two having different characteristics and requirements, with the storefront mostly about read traffic. The checkout covers most of the writing and interacts with external systems. Shopify supports various payment systems. According to de Water, Shopify’s main challenge is dealing with flash sales and some weekend sales. To show what scalability issues they are dealing with, de Water said, “Here are last year’s numbers. $6.3 billion total sales with requests per minute peeking up to 32 million requests per minute.” To check that everything works, the team performs load testing in production using Genghis, a custom load generator that creates worker VMs that execute Lua scripts describing the behavior that they want and a resilience matrix simulating an outage to see what happens when one would take place, and what needs to be done to prevent one in production. They also use a circuit breaker to protect datastores and APIs, activating it when necessary. At the end of the talk, de Water answered specific questions on handling big clients, moving to microservices, pods, scaling the storefront and checkout, using Packwerk to enforce boundaries between teams, and others. SPONSORED NGINX [NGINX]( [97 Things Every SRE Should Know]( Code Climate [Code Climate]( [The Engineering Leader's Guide to Cycle Time]( Debug, Analyze, And Optimize... In Production! An Observable Service with No Logs by [Glen Mailer](, Senior Software Engineer @Geckoboard Mailer talked about a microservices project written in Go that they did at CircleCI. They needed a new service for a build agent, and decided to build it in Go (rather than Clojure, as many other services had been). They also wanted a new approach to logging, one that was not based on text logs but rather used what he called an “event-first approach,”—a data-first telemetry instead of a text-first telemetry as it is for regular logs. What takes place in a system is described through events with each having a unique name throughout the codebase identifying what takes place in the system. The properties of the respective event are structured data. The resulting log lines are made of structured data. He then showed us some Go code on how this event-first observable service is written, mentioning the patterns that emerged while developing it and how they implemented them. Also, he showed the interface built on top of it, using it to visualize logged data and interpret it. The project started as an experiment, a partial success according to Mailer. He learned a lot about what was important to log. Having a new approach to logging was, in his opinion, much better than logging some text lines when some code gets executed. What Mailer didn't like about the result was that it was like “reinventing a bunch of wheels.” He still considered the process valuable for their team because they discovered together what to include and exclude from the library they had built up. Concluding, Mailer “tentatively recommends this concept of a unified telemetry abstraction, having this one place to produce logs, metrics, and events.” Still, he had some concerns because others did not find the new service as intuitive as hoped. He considered the new logging service good, but not good enough to rip the present system apart and start from scratch. He recommended looking at OpenTelemetry, an open-source project somewhat similar to theirs. Profiles, the Missing Pillar: Continuous Profiling in Practice by [Michael Hausenblas](, Solution Engineering Lead @AWS Hausenblas discussed the need for observability in large, sometimes distributed systems. He defined observability as “the capability to continuously generate and discover actionable insights based on certain signals from the system under observation.” He mentioned several types of signals: text logs, usually consumed by humans, then metrics, labeled as numerical signals, and distributed traces used for “propagating an execution context along a request path in a distributed system.” He then added another signal type, namely profiles which he saw in the context of the source code and the running process associated, exploring them to find out more about the system monitored. Unlike metrics, which have their benefits, profiles allow going to a specific line of code that is responsible for a certain metric that is out of normal values. He also mentioned 3 open source continuous profiling tools that developers can install and use right away: - Parca—inspired by Prometheus. - Pyroscope—a rich tool with broad platform support that supports several language profiles and eBPF. - CNCF Pixie from the Cloud Native Computing Foundation donated by New Relic, currently a sandbox project. It uses eBPF for data collection. Hausenblas predicted that eBPF will become the “unified collection method for all kinds of profiles.” All profiling tools recommended by Hausenblas use eBPF. Hausenblas concluded his session by answering questions on Profiling and the Development Life-Cycle, differences between eBPF and language-specific tools, and the missing piece in Continuous Profiling. Highlighting the benefits of using eBPF, he said that eBPF does not care if the program was written in a compiled or interpreted language. Also, it captures both function calls and syscalls, all in one place. SPONSORED Tonic.ai [Tonic.ai]( [The Subtle Art Of Giving A F*** About Data Privacy]( MariaDB [MariaDB]( [Cloud Architecture Checklist]( Effective Microservices: What It Takes To Get The Most Out Of This Approach Airbnb at Scale by [Selina Liu](, Senior Software Engineer @Airbnb Liu shared lessons learned at Airbnb during their migration from a monolith to SOA. The first lesson was to invest early in the needed infrastructure because that will “help to turbocharge your migration from monolith to SOA.” They approached the migration by decomposing the presentation layer into services that “render data in a user-friendly format for frontend clients to consume.” Below those services, they have mid-tier services with various business rules which use data services connected to databases to access data uniformly. After migration, their application was made up of 3 layers: presentation, business, and data running as services. The higher layer may call a lower layer but not the other way around. Liu then explained using a concrete case how they migrated Host Reservation Details, a certain feature they used, to SOA, as an example to show how they tackled the migration process. To help with the migration process, they used an API Framework used by all Airbnb services to talk to each other. Then they had a multi-threaded RPC client handling features like error propagation, circuit breaker requests, and retries. That helped engineers focus on the business logic to create new functionality rather than inter-service communication. After that, she talked about Powergrid, a Java library used to run code in parallel. Powergrid “helps us organize our code as a directed acyclic graph, or a DAG, where each node is a function or a task.” This library enables them to perform low-latency network operations concurrently. Then she mentioned OneTouch, a “framework and a toolkit built on top of Kubernetes that allows us to manage our services transparently and to deploy to different environments efficiently.” One last piece of useful infrastructure she mentioned was Spinnaker, defined as an “open source continuous delivery platform that we use to deploy our services. It provides safe and repeatable workflows for deploying changes to production.” These pieces of infrastructure helped them migrate their monolith to 400+ services with over 1000 deploys daily over 2-3 years. She also mentioned that “as we continue to evolve our SOA, we also decided to unify our client-facing API. Our solution to the problem is UI Platform, a unified, opinionated, server-driven UI system.” As another conclusion, she added that having the UI generated with the backend services allows them to have a “clear schema contract between frontend and backend, and maintain a repository of robust, reusable UI components that make it easier for us to prototype a feature.” Liu’s session ended by answering questions, one of them regarding the use of SOA instead of microservices. She said “I think, tech leaders at that time, they really just used the word SOA to push the concept wider to every corner of the company. I think partially it might be because, at the start, we were wary of splitting our logic into tiny services. We wanted to avoid using microservices, just because the term might be a little bit misleading or biased in just how it’s understood by most people when they first hear it. It’s probably just like the process of how we communicate it, more than anything else.” Liu also talked a bit about the technologies used and the benefits resulting from migration. SPONSORED Microsoft [Microsoft]( [Introduction to Java on Azure]( Hasura [Hasura]( [2022 Trend Report: Enterprise Application Integration (By DZone)]( Pump It Up! Actually Gaining Benefit from Cloud Native Microservices by [Sam Newman]( In this session, Sam Newman discussed cloud and microservices, what to do about them to succeed, and shared some tips on how they should be done and certain pitfalls to avoid. He started by wondering why cloud-native and microservices have not delivered as expected. Part of the unsuccessful pairing of cloud and microservices is, in his opinion, a certain level of confusion on the role of the cloud in an organization. He tells the story of visiting a company in Northern England where developers said they wanted access to the cloud. When he told their desire to the CIO, he discovered they already had access to the cloud. It turned out that if a developer wanted access, he had to create a JIRA ticket and someone from the system administration would intervene and provide the needed access. It was like using a resource on premises in the old way but now in the cloud. This made the process inconvenient for developers, slowing the entire development process. Talking about the revolution introduced by AWS with cloud computing, Newman said “AWS’s killer feature wasn’t per-hour rented managed virtual machines, although they’re pretty good. It was this concept of self-service. A democratized access via an API. It wasn’t about rental, it was about empowerment.” He continued inviting leaders to trust their people, adding “this is difficult because, for many of us, we’ve come from a more traditional IT structure, where you have very siloed organizations who have very small defined roles.” To address this issue he suggested leaders think in terms of long-lived product teams who own major parts of the project—think “holistically about the end-to-end delivery of functionality to the users of your software.” That’s why Newman considered microservices to be of particular value because teams can own their microservice. Besides this ownership issue, Newman pinpointed the lack of proper training in using the tools they have as a pitfall to a project’s success. He considered it not enough to throw some tools at them but follow up with the necessary training on using those tools. Going further, he advised creating a good developer experience by treating the microservices platform as a product, talking to developers, testers, and security personnel to find out what they need to succeed, to create an “experience for your delivery teams that makes doing the right thing as easy as possible,” making developers, testers, sysadmins your own customers. SPONSORED ING [ING]( [The ING Blog - Explore Articles on Domain Driven Design, Architecture, and more]( DeepGram [DeepGram]( [Speech Recognition with Twilio and Python](   Follow InfoQ on: [Twitter]( [Facebook]( [Youtube]( [LinkedIn]( You have received this message because you are subscribed to the “Special Reports Newsletter”. To stop receiving this email, please click the following link [Unsubscribe]( - - - C4Media Inc. (InfoQ.com), 2275 Lake Shore Boulevard West, Suite #325, Toronto, Ontario, Canada, M8V 3Y3