Midjourney on Google Cloud, Online Model Serving & Reduction Server on Vertex AI

DataPro#35: PaLM-E: An embodied multimodal language model                                                                                                                                                                                                                                                                                                                                                                                                                 [Open in app]( or [online]() [Midjourney on Google Cloud, Online Model Serving & Reduction Server on Vertex AI]( DataPro#35: PaLM-E: An embodied multimodal language model Mar 17   [Save]( [▷  Listen](   👋 Hey, "Artificial intelligence will reach human levels by around 2029. Follow that out further to, say, 2045, we will have multiplied the intelligence, the human biological machine intelligence of our civilization, a billion-fold." - [Ray Kurzweil, Computer Scientist, Author, Inventor, and Futurist.]( Despite our great progress in AI & ML, we still have a long way to go! In this week's edition, we’ll explore recent advancements in critical techniques to help us address bottlenecks and improve our productivity. Key Highlights: - [Implementing online model serving]( - [Building the most open and innovative AI ecosystem]( - [PaLM-E: An embodied multimodal language model]( If you are interested in sharing ideas and suggestions to foster the growth of the professional data community, then this survey is for you. Consider it as your space to share your thoughts! Jump on in! [TELL US WHAT YOU THINK]( Cheers, Merlyn Shelley Associate Editor in Chief, Packt Latest Forks on GitHub - [SeldonIO]( MLOps platform for deploying, monitoring, and managing ML models at scale. - [pytorch]( Serve, optimize and scale PyTorch models in production. - [PaddlePaddle]( Deploy AI models easily and quickly on Cloud, Mobile, and Edge. - [bentoml]( This lets you deploy, operate and scale Machine Learning services on Kubernetes. - [openai]( Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks. - [tesseract-ocr]( This package contains an OCR engine - libtesseract and a command line program - tesseract. - [ultralytics]( Future vision AI research, open-source, with evolved best practices. Industry Insights AWS - [Bring legacy machine learning code into Amazon SageMaker using AWS Step Functions:]( This post discusses migrating legacy ML code to the AWS Cloud using [Amazon SageMaker]( and [AWS Step Functions](. Legacy code is not built for cloud-ready SDKs like [AWS SDK for Python]( or [SageMaker Python SDK]( but this solution requires minimal code refactoring and can be extended for additional functionality. It shows how data scientists and MLOps engineers can collaborate to migrate many legacy models. - [Monitoring load balancers using Amazon CloudWatch anomaly detection alarms:]( The post discusses monitoring the [AWS Network Load Balancer]( (NLB) metric TCP_Target_Reset_Count and explains why conventional [Amazon CloudWatch]( alarms using static thresholds cannot be used. It explores using CloudWatch anomaly detection alarms for this purpose and highlights situations where this type of monitoring can be useful. - [Few-click segmentation mask labeling in Amazon SageMaker Ground Truth Plus:]( This post discusses building an ML model for semantic segmentation, which requires labeling a large volume of data at the pixel level. An extension to the Ground Truth auto-segment feature is introduced that enables corrective clicks to guide the ML model for better predictions. A basic architectural pattern is presented for integrating interactive tools into Ground Truth labeling UIs. Google Cloud - [Optimize PyTorch training performance with Reduction Server on Vertex AI:]( The use of distributed training is becoming essential for deep learning models due to their increasing complexity and larger datasets. This post focuses on how [Reduction Server]( a feature of Vertex AI, can optimize bandwidth and latency for multi-node distributed training on NVIDIA GPUs, resulting in faster training for PyTorch + Hugging Face models. - [Building the most open and innovative AI ecosystem:]( Google Cloud has announced that three more companies, [AI21Labs]( [Midjourney]( and [Osmo]( are joining its platform to build and train foundation models and generative AI. The goal is to offer developers and partners access to innovative AI and ML tools, including new generative AI capabilities. Additionally, Google Cloud is launching a new initiative, [Built with Google Cloud AI]( to support its partners and accelerate app development. - [How Osmo is digitizing smell with Google Cloud AI technology:]( Osmo is working with Google Cloud's AI technology to digitize and map our sense of smell. By training models on Vertex AI and using Dataflow to analyze large datasets, they aim to create more sustainable aroma molecules and improve human health and well-being. They are building foundational capabilities to enable computers to replicate our sense of smell. Reading from the UK or the US? Check out our offers on[Amazon.com]( [Amazon.co.uk]( Just for Laughs! Why did the deep learning model go to the gym? Because it wanted to improve its weights and biases! Understanding Core Concepts Implementing Online Model Serving – By [Md Johirul Islam]( Let’s train a dummy SGDRegressor model, use Flask to create a server and API for the online prediction endpoint, use Postman to send a request to the server, and update the model with the input data and the prediction made by the last model. For the end-to-end example we are going to run, you need to import the following modules: from flask import Flask, request import numpy as np import json from sklearn.linear_model import SGDRegressor Let’s begin: - First of all, let’s create a model with some dummy data in the following code snippet: X = [ [1, 1, 1], [1, 1, 1], [1, 1, 1], [2, 2, 2], [2, 2, 2], [2, 2, 2] ] y = [1, 1, 1, 2, 2, 2] model = SGDRegressor() model.fit(X, y) print("Initial coefficients") print(model.coef_) print("Initial intercept") print(model.intercept_) We notice that the model is trained with the X and y dummy data. We get the initial coefficients and the intercepts as the following: Initial coefficients [0.29263344 0.29263344 0.29263344] Initial intercept [0.16956488] - Now we create the server using Flask and a prediction API using the following code snippet: app = Flask(__name__) def update_model(Xn, yn): print("Updating the model") model.partial_fit(Xn, yn) print("New coefficients now") print(model.coef_) print("New intercept now") print(model.intercept_) @app.route("/predict-online", methods=['POST']) def predict_online(): predictions = [] X = json.loads(request.data) print("Input data is ", X) predictions = model.predict(X) update_model(X, predictions) return json.dumps(predictions, cls=NumpyEncoder) if __name__ == "__main__": app.run() We have the update_model function, which takes new data and updates the model using the model.partial_fit(..) function. We use Flask just for the demonstration purpose of creating client APIs. You might be interested in using the upgraded FastAPI python web framework to create the APIs. - Now, we go to Postman and call the predict-online API - If we check the console now, we will notice that the print statement inside the update_model function has appeared. From this execution, we get the following output: Updating the model New coefficients now [0.29263309 0.29263309 0.29263309] New intercept now [0.16956489] In this section, we have walked through a basic dummy end-to-end example of serving a model using the online serving pattern. You can take this workflow to serve any real-life model where the online serving pattern is a suitable solution. This content is curated from the book, [Machine Learning Model Serving Patterns and Best Practices (packtpub.com).]( To learn more, click on the button below. [SIT BACK, RELAX & START READING!]( Find Out What’s New? - [PaLM-E: An embodied multimodal language model:]( [PaLM-E]( is a new generalist robotics model that uses transfer learning from visual and language domains to overcome the lack of large datasets in robotics. It combines [PaLM]( a powerful large language model, with sensor data from the robotic agent to become a competitive model in the visual-language domain. - [How to Evaluate the Quality of Python Packages:]( This tutorial explains how to evaluate the quality of third-party Python packages before using them in a project. It recommends checking the Python Package Index, Libraries.io, GitHub repository, and license of a package to avoid incompatible or harmful code. - [The Difficulties of Monitoring Machine Learning Models in Production:]( As a data scientist, the challenge is to monitor three critical components: code, data, and the model. Each presents unique challenges, making production monitoring difficult. Ensuring a smooth operation of the entire model and pipeline is a key responsibility. - [Using MLflow with ATOM to track all your machine learning experiments without additional code:]( [MLflow Tracking]( is an API and UI that logs parameters, metrics, and output files during machine learning experiments. [ATOM]( is an open-source Python package that enables data scientists to explore machine learning pipelines by tracking models, parameters, pipelines, data, and plots. - [Implementing Deep Convolutional Neural Networks for QR Code-Based Printed Source Identification:]( The study compared several pre-trained CNN models such as AlexNet, DenseNet201, GoogleNet, MobileNetv2, ResNet, VGG16 for their ability to predict the source printer of QR codes accurately. A customized CNN model performed better in identifying printed sources of grayscale and color QR codes with less computational power and training time. See you next time! As a GDPR-compliant company, we want you to know why you’re getting this email. The _datapro team, as a part of Packt Publishing, believes that you have a legitimate interest in our newsletter and its products. Our research shows that you opted-in for email communication with Packt Publishing in the past and we think your previous interest warrants our appropriate communication. If you do not feel that you should have received this or are no longer interested in _datapro, you can opt out of our emails by clicking the link below.   [Like]( [Comment]( [Share](   Read Packt DataPro in the app Listen to posts, join subscriber chats, and never miss an update from Packt SecPro. [Get the iOS app]( the Android app]( © 2023 Copyright © 2022 Packt Publishing, All rights reserved. Our mailing address is:, Packt Publishing Livery Place, 35 Livery Street, Birmingham, West Midlands B3 2PB United Kingdom [Unsubscribe]() [Start writing]()