Using XAI, ChatGPT in Cloud & Feature Engineering Techniques

Packt DataPro#32: AWS and Hugging Face to make generative AI! [View this email in your browser]( [PacktDataPro Logo]( February 24, 2023 | DataPro #32 👋 Hey {NAME}, "AI can be a powerful tool for driving innovation and progress, but we must also consider the potential risks and challenges that come with its development." - [Jack Clark, OpenAI policy director]( AI may seem daunting with all its associated risks and challenges, but we must remember that every great invention comes with many challenges. So, let's embrace the potential of AI, and with careful empirical validation we can push innovation and progress to new heights. Today's edition will focus on exploring methods of incorporating XAI into our everyday activities, like driving cars, as well as examining its effects and strategies for addressing the challenges. Key Insights: - [Using XAI and ethics to control a decision tree]( - [How To Use ChatGPT in Cloud Computing]( - [Finding the Fastest Lane Using Machine Vision]( If you are interested in sharing ideas and suggestions to foster the growth of the professional data community, then this survey is for you. Consider it as your space to share your thoughts! Jump on in! [TELL US WHAT YOU THINK]( Cheers,Merlyn Shelley Associate Editor in Chief, Packt Keep up with cutting-edge Research on GitHub - [ControlNet:]( ControlNet is a neural network structure to control diffusion models by adding extra conditions. - [mm-cot:]( Multimodal-CoT incorporates vision features in a decoupled training framework. - [BioGPT:]( Official implementation of [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](. - [stable-diffusion-webui:]( A browser interface based on Gradio library for Stable Diffusion. - [T2I-Adapter:]( Official implementation of “Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models”. - [mario-gpt:]( Code for the paper "MarioGPT: Open-Ended Text2Level Generation through Large Language Models". - [ColossalAI:]( Colossal-AI provides a collection of parallel components that assists in writing distributed deep learning models. - [lion-pytorch:]( Evolved Sign Momentum, a new optimizer discovered by Google Brain that is purportedly better than Adam(w), in Pytorch. [EMAIL FORWARDED? JOIN DATAPRO HERE!]( Stay informed about Data & ML Industry Insights AWS - [MLOps deployment best practices for real-time inference model serving endpoints with Amazon SageMaker:]( [Amazon SageMaker for MLOps]( streamlines the ML lifecycle with tools to standardize and automate processes. The platform includes advanced deployment patterns for managing new models, with [deployment guardrails]( and [production variants]( capabilities. This post covers the deployment of ML models using [SageMaker]( in a repeatable and automated fashion, integrating MLOps tools and deployment patterns. It primarily focuses on real-time single-model endpoints, providing an overview of integrating SageMaker MLOps tools with the platform. - [AWS and Hugging Face collaborate to make generative AI more accessible and cost efficient:]( [Amazon SageMaker]( is a suite of tools and AI services that includes [Amazon CodeWhisperer]( which generates code recommendations based on code and comments. AWS has also created specialized ML accelerators for training and inference, called [AWS Trainium]( and [AWS Inferentia](. [Hugging Face]( has chosen AWS because it provides flexibility in using state-of-the-art tools for training, fine-tuning, and deploying Hugging Face models. This makes it easier and faster for developers to optimize performance and lower costs in producing generative AI applications. Google Quantum AI - [Suppressing quantum errors by scaling a surface code logical qubit:]( Soon, fault-tolerant quantum computers will be used for large-scale computations with applications in every industry. These computers will consist of millions of qubits, which must be good enough to avoid errors. Target error rates will be used to measure progress toward building fault-tolerant quantum computers. Improvements will lead to entering the fault-tolerant regime, where logical errors can be exponentially suppressed, and the first useful error-corrected quantum applications can be unlocked. - [FRMT: A benchmark for few-shot region-aware machine translation:]( [FRMT dataset]( and evaluation code have been released to enable researchers in creating machine translation systems for various regional languages spoken across the globe. The goal here is to generate language appropriate for users' locality or region. The dataset allows the research community to [compare performance]( for region-aware MT models, validated with thorough human-evaluation studies. The language varieties in FRMT have significant differences that output from region-aware MT models should reflect. Just for Laughs! Why did the self-driving car cross the road? To get to the other side of the neural network! Understanding Data & ML Core Concepts Using XAI and ethics to control a decision tree - By [Denis Rothman]( We will discuss here how to reduce errors when using the autopilot mode on self-driving cars (SDC). To do this we will be introducing real-life bias, moral, and ethical issues in the decision tree to measure SDC reactions. Loading the model # Applying the model # Load model dt = pickle.load(open('dt.sav', 'rb')) Do note, no matter what trained autopilot AI algorithm you load there are some situations that require more than just mathematical responses. Accuracy measurements t = 0 # true predictions f = 0 # false predictions The model encounters f1 and f2, the right lane features. The model encounters f3 and f4, the left lane features. The decision tree decides which lane provides the highest level of security no matter what the situation is. Can you allow the SDC to kill a pedestrian even if the prediction is true? Let's explore real-time cases and see what can be done. Simulating real-time cases We have loaded the model and have our two measurement values. We cannot trust these measurement values anymore. Let’s now examine the input data line by line. Each line provides the four features necessary to decide to stay in a lane or swerve over to another lane. for i in range(0, 100): xf1 = pima.at[i, 'f1'] xf2 = pima.at[i, 'f2'] xf3 = pima.at[i, 'f3'] xf4 = pima.at[i, 'f4'] xclass = pima.at[i, 'label'] Introducing ML bias due to noise Bias comes from many factors, such as: - Errors in the algorithm when facing new data - A sudden shower (rain, snow, or sleet) that obstructs the SDC's radar(s) and cameras - The SDC suddenly encounters a slippery portion of the road (ice, snow, or oil) - Erratic human behaviors on the part of other drivers or pedestrians - Windy weather pushing objects on the road - Other factors Any or all of these events will distort the data sent to the autopilot's ML algorithm(s). Here, we will introduce a bias value for each feature: b1 = 1.5; b2 = 1.5; b3 = 0.1; b4 = 0.1 These bias values provide interesting experiments for what-if situations. You can modify the value and security level of one or several of the features. The values are empirical. The program will then apply the simulation of real-life data distortions to the input data: xf1 = round(xf1 * b1, 2) xf2 = round(xf2 * b2, 2) xf3 = round(xf3 * b3, 2) xf4 = round(xf4 * b4, 2) The program is now ready to make decisions based on real-life constraints. The program compares the distorted data prediction with the initial training data: X_DL = [[xf1, xf2, xf3, xf4]] prediction = dt.predict(X_DL) e = False if (prediction == xclass): e = True t += 1 if (prediction != xclass): e = False f += 1 Introducing ML ethics and laws Case 1 – not overriding traffic regulations to save four pedestrians. Now we add a constraint. In this situation, if you change the values of the b parameters the positive ethical bias scenario could be: b1 = 1.5; b2 = 1.5; b3 = 0.1; b4 = 0.1 Case 2 – overriding traffic regulations. The following scenario increases the value of the left lane values authorizing the autopilot to override traffic regulations: b1 = 0.5; b2 = 0.5; b3 = 1.1; b4 = 1.1 Case 3 – introducing emotional intelligence in the autopilot. We will now add a kill switch to our code when the sum of the bias parameters is too low for the global traffic in a geographical zone: if float(b1 + b2 + b3 + b4) <= 0.1: print("Alert! Kill Switch activated!") break When the traffic is too dense ahead, the autopilot should provide alert bias values to the ML algorithm: b1 = -0.01; b2 = 0.02; b3 = .03; b4=.02 An alert is now activated well ahead of the heavy traffic zone or location with too many pedestrian crossings: Alert! Kill Switch activated! true: 1 false 0 accuracy 1.0 We have implemented a decision tree in an SDC's autopilot and provided it with traffic constraints. We enhanced the autopilot with an ethical kill switch teaching a machine to "fear" the law. This content is curated from the book, [Hands-On Explainable AI (XAI) with Python (packtpub.com).]( To learn more, click on the button below. [SIT BACK, RELAX & START READING!]( Find Out What’s New in Data & ML - [How To Use ChatGPT in Cloud Computing:]( By integrating ChatGPT into [cloud-based collaboration tools]( could streamline the process of summarizing longer documents and reducing the workload of employees. Also, ChatGPT could be used to summarize customer service tickets and previous interactions with customers, assisting customer service representatives with more context. - [The public supports regulating AI for safety:]( The US-based [Monmouth University]( recently conducted an analysis of the Impact of AI in society. It clearly shows that people are deeply concerned about the potential effects of AI's integration on their daily decision-making processes, whether in the workplace or in everyday life. - [The Impact of AI-enabled Data Analytics Services Across Major Industries:]( Enterprises are using AI and ML to automate manual processes in customer data analytics, including data cleaning, feature selection, and model selection. These technologies can uncover patterns and relationships that are difficult for humans to identify. AI helps organize and integrate data with platforms and predict strategic directions for business growth and ROI maximization. - [Tutorial: Linear Functions in Machine Learning:]( Linear regression is a popular tool in data science because it matches well with human intuition, and changes in predictors produce proportional changes in outcomes. This tutorial explores the [Automobile Data Set]( from the UCI Machine Learning Repository for applying linear regression to predict the risk associated with a car based on engine size and horsepower. - [Finding the Fastest Lane at Border Crossings Using Machine Vision:]( This post discusses how OpenCV and YOLOv3 can be used to detect and track moving vehicles in real-time. The technology can help save time and avoid long queues at border crossings by using machine vision algorithms to determine vehicle speed and traffic volume. The code is [available on GitHub]( for anyone to experiment. - [Boost Machine Learning Model Performance through Effective Feature Engineering Techniques:]( The article aims to improve credit card fraud detection using machine learning models ([XGBoost]( by implementing feature engineering techniques. It analyzes the difference in model performance before and after feature engineering to highlight its significance in obtaining accurate predictions. See you next time! Clicking unsubscribe will stop all [_datapro]( communication. Make sure you don't make a hasty decision! [NOT FOR YOU? UNSUBSCRIBE HERE]( [Facebook icon] [Instagram icon] [Twitter icon] [Logo] Copyright (C) 2023 Packt Publishing. All rights reserved. Hello, Thank you for being a part of the DataPro weekly newsletter. Team Packt. As a GDPR-compliant company, we want you to know why you’re getting this email. The _datapro team, as a part of Packt Publishing, believes that you have a legitimate interest in our newsletter and its products. Our research shows that you,{EMAIL}, opted-in for email communication with Packt Publishing in the past and we think your previous interest warrants our appropriate communication. If you do not feel that you should have received this or are no longer interested in _datapro, you can opt-out of our emails by clicking the link below. Our mailing address is: Packt Publishing Livery Place 35 Livery StreetBirmingham, West Midlands B3 2PB United Kingdom [Add us to your address book]( Want to change how you receive these emails? You can [update your preferences]( or [unsubscribe](