Containers Are The Present, Not the Future, Get Back to Basics, and a Brief History of Machine Learning (Database Weekly 3/30/2019)

 [Database Weekly]( The Complete Weekly Roundup of SQL Server News Hand-picked content to sharpen your professional edge Editorial  Containers Are The Present, Not the Future Last week was the MVP Summit. This year included a bunch of very technical discussions about some of the future of the Microsoft Data Platform (a big thank you goes to Slava Oks, Bob Ward, and all the team for a great job). I can’t share much since it was all under NDA. All I can really say about it is that there are some cool things coming. Fortunately, there are a couple of technologies that are clearly going to have a huge impact on the future of the Data Platform that I can share with you. These technologies are released, and people are already working with them in ways big and small, from development to production. They are notebooks and containers. I’m just barely getting the concepts behind notebooks and the opportunities that they’re going to offer into my tiny brain. So, I’ll just link you to this excellent [blog post]( by Gianluca Sartori on putting Glen Berry’s DMV scripts into a Notebook. It’s a glimpse of what’s possible. However, I am able to talk about containers. SQL Server has been running on containers for at least 18 months at this point. I’ve been using them myself for just shy of a year. If you have yet to start to explore the concepts around containers and container orchestration systems like Kubernetes, it’s past time for you to get started. This technology is going to disrupt the way we do our jobs in a very big way. I’ve heard them compared to the disruption caused by virtual machines, but containers go way beyond that. What you can do with this technology just changes what we can do, more and faster than just about anything I’ve seen in my 30 years in tech (TCP/IP being released publicly was bigger, but this is big). What I find most fascinating about containers though is not all the possibilities (and have a chat with the people who’ve been working with them for a while to get some idea of how huge this is, I recommend Anthony Nocentino or Rob Sullivan). No, instead, it’s the resistance. Many people, who have never worked with containers, who haven’t studied them, who have only heard others talking about them, are incredibly quick to say, “It’ll never work.” I’ve heard these voices before. I remember when I first saw a good demo of virtual machines. It was fascinating, except I heard all these people saying “We’ll never have our database in production in a virtual machine. It’ll never scale/perform/availability/something as well as our hardware.” Yet, today, most people are running part, or even all, of their load on virtual machines, in production, and it works fine. The first time I saw Azure SQL Database, I was blown away by the possibilities (and bummed by some of the insane limitations, 2gb, really?). However, behind me, I heard, over and over, “We’ll never have a production system on this because of scale/performance/availability/something.” Yet, today, again, more and more people have at least a hybrid system, split between the cloud and on-premises. I even had one organization tell me that they were getting better performance in Azure SQL Database than they used to get locally (and yeah, maybe there’s a long discussion to be had there). My point is, before you start saying “never” and complaining about how this technology doesn’t work EXACTLY like the technology you’re used to using, let’s try something different. Instead, it’s probably time to start to look at new technology as what it is, opportunity. There’s a chance that this stuff will fix a problem you have today. There’s a chance this stuff will prevent a problem you’re going to face tomorrow. It’s time to recognize that as a technologist, and if you’re developing, administering, maintaining, architecting databases, that’s what you are, you need to embrace change and get on board with it. Learn this stuff now, while it’s just getting going. That will put you ahead of the curve, but it will also enable you to help your organization to move forward. Trust me, just like virtual machines, just like cloud technologies, containers are coming to your organization. Do you want to be remembered as the speed bump that slowed down successful implementation, or the facilitator that helped to make it happen? Which one do you think pays better? Grant Fritchey [Join the debate, and respond to the editorial on the forums](  [Redgate Webinars]( The Weekly News All the headlines and interesting SQL Server information that we've collected over the past week, and sometimes even a few repeats if we think they fit. Administration [Accelerated Database Recovery: How Big Is My Version Store?]( When Query Store rolled out, there were a lot of questions about controlling the size, and placement of it. To date, there’s still not a way to change where... [Adding a New Transaction Log File when the Existing Log is Full]( SQL Server Transaction Log files store details about every modification made to a SQL Server Data File. This includes when you need to add a 2nd log file. The... [The SQL Server Data Row Size Question: Why is It Bigger?]( This morning, I received the following question from a user: Hello Madam, Could you please clarify SQLServer “Data Row” size:If I run the script below on SQL Server 2012,... [What Are You Going to Do About SQL Server 2008?]( With support ending soon, Brent Ozar wonders what all the companies still running SQL Server 2008 are planning to do with those instances. [Scripting Out a SQL Server instance (Agent jobs, XEvents, Triggers and the like)]( In order for a database system to work, you often need to provide programmable server objects. I’ve written very few databases that didn’t include agent jobs and triggers, or... [Oracle RAC vs. SQL Server AG]( As I have seen the benefit for having a post on Oracle database vs. SQL Server architecture, let’s move onto the next frontier- High Availability…or what people think is... Azure SQL database [Enterprise-scale Backup for SQL Server Databases]( More and more organizations are moving to the cloud to leverage the key benefits of agility and reduced overall cost. Its likely that at some point in time, your... Big Data [Databricks Dashboards: Data Exploration With Salary Classification]( For many companies, the initial attraction to Azure Databricks is the platform’s ability to process big data in a fast, secure, and collaborative environment. However, another highly advantageous feature... [Generate Big Datasets with Hive in HDInsight]( TThis post describes how to generate big datasets with Hive in HDInsight, specifically TPC-DS benchmarking datasets. There are many tools for generating sample data, and this one is particularly... DBA Tools [Connect To SQL Server – Back to Basics]( The first critical task any data professional should ever learn how to do is how to connect to SQL Server. Without a connection to SQL Server, there is barely... [Generating a Jupyter Notebook for Glenn Berry’s Diagnostic Queries with PowerShell]( The March release of Azure Data Studio now supports Jupyter Notebooks with SQL kernels. This is a very interesting feature that opens new possibilities, especially for presentations and for... Data Science [Machine Learning Digs Deeper into Azure with Custom Vision, Anomaly Detector]( Microsoft's quarter of product announcements underscores its commitment to the cloud, its investment in machine learning as a core component of the modern enterprise, and an expansion of its... Database Design [Graph Databases: They Who Forget the Past…]( Out of the plethora of misconceptions common in the industry, quite a few are squeezed into this paragraph "The relational databases that emerged in the 80s are efficient at... Devops and Continuous Delivery (CI/CD) [Approaching Zero: Wrapping Up]( This is part nine in a series on near-zero downtime deployments. What Have We Learned? [4 top tips for starting your database DevOps journey]( If you’ve read the Accelerate State of DevOps Report from DORA and the 2019 State of Database DevOps report from Redgate, you’ll see how they resonate in terms of the benefits to be gained from adopting DevOps. [How to get Database DevOps implemented at your business]( Tuesday April 23, 11 AM-12 PM Pacific / 1 PM-2 PM Central - Join Microsoft Data Platform MVP Kendra Little to dive into the key findings of Redgate’s 2019 State of Database DevOps Report and investigate the growing importance of the database in successful DevOps and IT performance. [The Problem with Continuous Delivery: Defining Continuous]( The IT community should spend time thinking about what continuity actually means in the context of continuous delivery. ETL/SSIS/Azure Data Factory [SSIS Design Pattern: Use Script Tasks for ETL Instrumentation]( I consider scripting in SQL Server Integration Services (SSIS) my data integration “get out of jail free” card. [Redgate SQL Change Automation]( Machine Learning (ML) [A Brief History of Machine Learning]( Machine Learning (ML) is an important aspect of modern business and research. It uses algorithms and neural network models to assist computer systems in progressively improving their performance. Microsoft News [The first cars using Microsoft Connected Vehicle platform at scale are coming]( In September 2016, Renault-Nissan announced it planned to use a number of Microsoft cloud services in its next-gen vehicles. This week, the auto maker is ready to deploy services... Performance Tuning [How to Determine Activity on Each Processor in SQL Server]( The contents of the SELECT, WHERE and ORDER BY clauses are all a matter of determining the story you need to tell. [Tracking the number of active sessions on a database using SQL Monitor]( IIt’s surprisingly useful to know the number of active sessions on each of the databases on your servers. With a bit of SQL, we can create a custom metric... [Can You Force A Parallel Plan in Query Store?]( I love the questions I get when presenting: Can You Force a Parallel Plan in Query Store. I haven’t a clue. The trick I think is going to be... [An Experiment with Deadlocks]( Everything can be fixed with a query hint (*cough* directive), right? If a certain process is consistently causing deadlocks, a simple ROWLOCK hint can be added to prevent it, right? [Comparing plans in Management Studio]( Previously Gail Shaw looked at using Query Store to compare execution plans, but it’s not the only way that two execution plans can be compared. PowerPivot/PowerView/PowerQuery/PowerBI [Affordable Power BI Premium for Small Business (Part 1)]( This is another one of those articles that has been swirling in my head for a very long time, about 2 years actually.  I have long been frustrated by the lack of a suitable scalable Power BI pricing model for small to medium sized enterprises... [Power BI Support, new workspaces, Power BI Desktop update and more… (March 25, 2019)]( This week's Power BI news roundup! [Power BI Sentinel]( We all know and love Power BI, because of it’s power and ease of use, there’s a good reason why it’s number 1 in the Gartner Quadrant. [Performance Design Tip for Power BI: Be Careful of Dates]( I have written about Date Dimension in Power BI in many posts, and it makes sense now to explain one of the most common performance challenges that I see... [Power BI, Caching, Parallelism And Power Query Refresh Performance]( Some time ago a customer of mine (thank you, Robert Lochner) showed me a very interesting scenario where a set of Power Query queries in Power BI Desktop refreshed... [Using DAX with DateTime values]( I was recently reading an interesting question by a user. Given a date/time value and a number of hours, add the number of hours to the date, but considering... Powerpivot [Three Ways to Use Power BI Dataflows]( Dataflows in Power BI are a really interesting capability for centralizing and reusing Power Query logic among many different PBIX files. SQL Server News [PASS turns 20]( Its really hard to believe that its been 20 years since the start of the PASS organization – the volunteer run database education group that I am proud to be associated with. My association runs to 18 of those 20 years, and 16 of those have been with... [Cumulative Update #14 for SQL Server 2017 RTM]( The 14th cumulative update release for SQL Server 2017 RTM is now available for download at the Microsoft Downloads site. [SQL Server 2019 community technology preview 2.4 is now available]( Microsoft have announced the release of SQL Server 2019 community technology Preview 2.4. [Getting Started with Azure Data Studio and SQL Server 2019 Preview]( SQL Server 2019 is in the final rounds of Community Technology Previews, and if... [How to Prepare for SQL Server 2008 and 2008 R2 End of Support]( SQL Server 2019 is in the final rounds of Community Technology Previews, and if I wasn’t excited about it before, I definitely am now. SQL Server on Linux [Installing and Configuring SQL Server on Linux]( I have shown you in the past how to install and run SQL Server in Docker Containers, and how to deploy Availability Groups and stand-alone SQL Server Instances... [SQL Server Data Pages on Linux]( One of the new phrases coming out of Microsoft is that “SQL is just SQL” regardless of what operating system it resides on. This was echoed during the keynote at SQL Bits 2019 by the Microsoft team, which you can watch here. T-SQL [Filling Missing ID Codes In SQL]( This is going to be a bit of a brain storming post that comes from an interesting question that I was asked today: “I’ve got a table with a ID code field, now some of the rows have a value in that field and some are NULL. How can I go about filling in ..." [Installation of SQL Server client tools]( It is the fourth post on series related to installation and configuration of SQL Server client tools required. This series is for professionals who start their journey with SQL Server administration and also for those who want to extend and structure... [How Does QUOTED_IDENTIFIER Work in SQL Server? – Interview Question of the Week #217]( It is the fourth post on series related to installation and configuration of SQL Server client tools required. This series is for professionals who start their journey with SQL Server administration and also for those who want to extend and structure... [Replacing SQL Cursors with Alternatives to Avoid Performance Issues]( In this article, we’ll look at some alternatives to using SQL cursors which can help to avoid performance issues caused by using cursors. Vendors/3rd-party Products [Refactoring a Batch or Object with SQL Prompt]( SQL Prompt is a tool I use all the time for code completion and code formatting tasks, to the point of it feeling very wrong to write a query without it. Write a SQL Statement with a JOIN, and it auto-suggests the ON criteria. [Simplify and improve your security model with SQL Census]( In the R&D division of Redgate, Foundry, we’re working on a new tool, SQL Census, in an effort to make your SQL Server permissions more manageable by seeing who has access to your servers and restructuring existing access rights into a simpler and more compliant format. [How are you using tools to design and model databases?]( Redgate invite you to participate in their survey on database modeling. In order to inform their future product decisions, they'd like to better understand the needs and motivations behind the use of such tools. Webinars [SQL in the City Summits coming this spring]( The SQL in the City Summits are back for 2019, and we’ve got more scheduled than every before. You can see the complete list on our event page, and I’m lucky to be invited to all of them. Here’s the schedule. [SQL in the City Streamed: Standardizing Database Change Management across your Organization]( Wednesday April 3, 09:00-14:00 Eastern / 14:00 - 19:00 BST - Join online for sessions designed to broaden your skillset, support your ongoing learning, and keep you up-to-date with the industry, presented by Steve Jones, Kendra Little, Grant Fritchey and Kathi Kellenberger. [Extending DevOps to the Database: Development Best Practice]( Wednesday April 10, 4PM-5PM BST / 10AM-11AM CDT - Discover how you can improve your database development and deployment processes. You'll see a demo of Redgate tools and see how they plug into your usual technology stack. There will also be plenty of time to ask Arneh questions. [How to Overcome the Top Blockers for Implementing Database DevOps]( Tuesday April 23, 11 AM-12 PM Pacific / 1 PM-2 PM Central - Join Microsoft Data Platform MVP Kendra Little to dive into the key findings of Redgate’s 2019 State of Database DevOps Report and investigate the growing importance of the database in successful DevOps and IT performance.  [RSS Feed]( This email has been sent to {EMAIL}. To be removed from this list, please click [here](. If you have any problems leaving the list, please contact the webmaster@sqlservercentral.com. This newsletter was sent to you because you signed up at SQLServerCentral.com. Note: This is not the SQLServerCentral.com daily newsletter list, and unsubscribing to this newsletter will not stop you receiving the SQL Server Central daily newsletters. If you want to be removed from that list, you can follow the instructions on the daily newsletter. ©2019 Redgate Software Ltd, Newnham House, Cambridge Business Park, Cambridge, CB4 0WZ, United Kingdom. All rights reserved. webmaster@sqlservercentral.com  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -