Working with an Azure Data Factory Meta-data Driven Pipeline (2021-11-22)

SQLServerCentral Newsletter for November 22, 2021 Problems displaying this newsletter? [View online](. [SQL Server Central]( Featured Contents - [Working with an Azure Data Factory Meta-data Driven Pipeline]( - [A Comparison of Database Drivers for MySQL]( - [Learn how to use Flyway for Automated Database Migrations]( - [Ransomware: A world under threat]( - [From the SQL Server Central Blogs - STRING_SPLIT Has Ordinals]( - [From the SQL Server Central Blogs - Decoding SSRS Subscription Agent Jobs]( Question of the Day - [The Row-Level Security Function]( The Voice of the DBA  Daily Coping Tip Connect with someone from a different generation I also have [a thread at SQLServerCentral]( dealing with coping mechanisms and resources. Feel free to participate. For many of you out there working in a new way, I'm including a thought of the day on how to cope in this challenging time from [The Action for Happiness Coping Calendar](. My items will be on [my blog](, feel free to share yours. The Complexity of Modern Systems I am a big advocate of version control and testing for your software. The main reason is that modern systems are incredibly complex, and capturing what has been done (and what is happening), along with looking for potential issues, is important. It might be critical for ensuring you have success in building software. Part of the reason is [this post on software complexity](. Now I don't think the complexity is killing software developers. I do think it adds to developers' stress, and in poorly managed environments, the complexity makes for a poor job experience for many. While I do think most people are smart, and they find ways to overcome this, they just don't enjoy the process. This is part of the reason, IMHO, that we are starting to see lots of people move to a new role, often in a new company. With remote work, there are more options, and for people with talent, they can easily leave. My recent job searches seem to show more remote job options and more demand for mid and senior level people. I think some of this is looking to get better talent in your company, and some are because experienced people are tired of poor work environments and poor code. Modern software is big, and I don't think any system of consequence is built by individuals anymore. There are still good projects and applications being built by a person, but once they start to grow, a team of developers is needed. This means that we have to learn how to better work together. We need to adhere to standards. We need to communicate and collaborate both in code as developers and together as humans. While someone may understand how the entire system works, it's unlikely that many of our staff does. If the system is large, it's unlikely anyone completely understands, and then our ability to function as a team is incredibly important to be sure that we are building something that works well together. To me, the complexity of modern systems emphasizes the need for teamwork and collaboration, the thing that many companies adopting DevOps struggle to emphasize. Steve Jones - SSC Editor [Join the debate, and respond to today's editorial on the forums](   Featured Contents [SQLServerCentral Article]( [Working with an Azure Data Factory Meta-data Driven Pipeline]( arindamxs from SQLServerCentral This article shows how to create a data driven ADF pipeline. [SQLServerCentral Article]( [A Comparison of Database Drivers for MySQL]( JerodJ from SQLServerCentral MySQL is "The world's most popular open source database" and it is widely-used to store and access your data. In fact, there are many cloud and on-premise databases (like MemSQL and Google Cloud SQL) that use the MySQL interface. Several native, open-source drivers are available for connecting to your MySQL data from other applications. In […] [External Article]( [Learn how to use Flyway for Automated Database Migrations]( Additional Articles from Redgate Here's your one-stop-shop for Flyway learning resources, curated by Tony Davis. Check out this handy learning pathway for managing and automating database deployments, from version control, using Flyway. [External Article]( [Ransomware: A world under threat]( Additional Articles from SimpleTalk Ransomware has threatened many organizations over the past few years. In this article, Robert Sheldon explains the history of ransomware and what needs to be done to protect against it. [Blog Post]( From the SQL Server Central Blogs - [STRING_SPLIT Has Ordinals]( Steve Jones - SSC Editor from The Voice of the DBA I noticed the docs for STRING_SPLIT now show that there is a third parameter. This brings us an ordinal for the positioning of the data. The format is now:... [Blog Post]( From the SQL Server Central Blogs - [Decoding SSRS Subscription Agent Jobs]( gbargsley from GarryBargsley Hello and welcome back… It has been a few months since I have published a blog post. More like 126 days, to be exact. Unfortunately, I have been dealing...   Question of the Day Today's question (by Steve Jones - SSC Editor):  The Row-Level Security Function What type of function is used in Row-Level Security? Think you know the answer? [Click here]( and find out if you are right.    Yesterday's Question of the Day (by Steve Jones - SSC Editor) Setting up a contained database I have just installed a new instance of SQL Server 2019 and want to run this code: CREATE DATABASE mytest CONTAINMENT = PARTIAL What should I do first? Answer: Change the instance "contained database authentication" to 1 Explanation: Before you can create a contained database, you need to enable "contained database authentication" at the instance level. Ref: Contained Database Authentication - [Discuss this question and answer on the forums](    Database Pros Who Need Your Help Here's a few of the new posts today on the forums. To see more, [visit the forums](. --------------------------------------------------------------- SQL Server 2016 - Administration [Turning Clustered index into PK without creating additional index?]( -  Adding PK to existing table like in the below example with existing Clustered index automatically creates an index on PK column. But I already have a CI index there, we don't want another one. is Turning Clustered index into PK without creating additional index possible via T-SQL? create table Z1 (id int not null, […] SQL Server 2016 - Development and T-SQL [Error handling/notification with a sql agent job]( - I have a stored procedure that is returning a set of data from a table and then is populating a summary table with that data. if sussful I am deleting all the data from the first table. if there is an error I do not want to delete the data and notify me if it […] [find which field and table blank or null field]( - I'm trying to turn this query into something that is more dynamic. I would like it to be able to grab a set of tables(example like sales%), and be able to tell me which table and field meets my criteria. then dump that record into some sort of generic error table. Thanks.  DECLARE @tb […] [Find the multiple records by max date]( - I have a table with multiple rows for a "Shelf Location" with the same date. example shelf 0095 has 10 records for 11/12/21 13:50:31.012 shelf 0095 has 15 records for 11/15/21 17:50:31.012 shelf 0145 has 5 records for 11/12/21 13:50:31.012 shelf 0145 has 3 records for 11/19/21 17:50:31.012 I want to select max date for […] SQL 2012 - General [How to add top 1 row as header on SQL server as extra row ?]( - I work on SQL server 2014 I face issue but I don't know how to solve it I need to add columns header as first row on table I Try as below but i get error Msg 213, Level 16, State 1, Line 1 Column name or number of supplied values does not match table […] SQL Server 2019 - Development [How to use a single stored procedure with one to 10 parameters]( - I would like to create one stored procedure that can handle from one to 10 parameters. Using the sledge hammer approach, I created 10 stored procedures; each one handles the number of parameters requested.  For example, if I wanted to look at the values for 1 tank, I would call the stored procedure that […] [Splitting one wide column into serveral different ones]( - Ahoi, i have a request to deal with that is in my opinion total bullshit, but thats besides the point. The data source provides me with a variable layout that depends on the column length (changing the type or structure of this column is out of question). This leads to the following issue: I have […] SQL Server 2008 - General [Sql 2008 export table csv with headers and data ...]( - Hi guys, I'm really a newbie in SQL, I hope you can help me with this. I need to export a table in SQL 2008 with headers and data delimited with | ( pipes ), but some information inside of this table have , ( comma ) and CR or LR, it's possible to put […] SQL Server 2008 Administration [system runs slowly, but SQL server can't use the RAM over 74GB?]( - My OS is windows server 2008 R2 Enterprise, SQL Serer is 2008 R2 Standard edition, the installed is 128GB, and RAM identified by windows is 128GB, there are about 100 users using system at the same time every day, many functions of system are running slowly, I set SQL Server maximum RAM 120GB, I observed […] SQL Azure - Development [Are there any read-made example databases I can use for a new Azure SQL Database?]( - (This post might be in the wrong place. Please forgive me if this is the wrong forum to ask this question.) I'm about to start writing a new application, to prototype some ideas. We want to have a database in Azure, so Azure SQL Database seemed the right choice, in this case. At this point […] Reporting Services [Report Viewer Paging Problem]( - Hiya, I'm using PowerBI on Premise Report Server and Visual Studio 2019. In development I can create reports that preview fine on separate pages. When they are promoted to live, the Report Viewer gets stuck on 0 pages and I cannot navigate them. After a while (several minutes) the Report Viewer sometimes comes to life […] [Is it possible to assign a random color to a font?]( - Hello, I am working on a report for which one of its requirements is that at the beginning of each year, the font color in one of its sections changes automatically. I am not sure if this is possible, I have search, found and read a few examples and they all have the colors to […] Integration Services [Can an SSIS package run without Integration Services being installed?]( - First off, its been over 12 years since being asked to deploy an SSIS package. I remember that we used to run them using dtexec. However, I'm not having much success. I've reviewed this article referenced in another thread. That's only 9 years old. I was optimistic that switching from the project deployment model to […] [Unicode data in sql server]( - Hi, I have a csv file which we export from Salesforce and flows to downstream sql server. So, the data flow is as follows: Salesforce --> csv --> Sql server The issue is that we have some unicode data in Name column ex; "Zuivelcoöperatie FrieslandCampina U.A. - NL" but while exporting to csv it comes […] Design Ideas and Questions [Help with star Schema]( - Hello I am just starting to learn about database designing, so as a start I am trying to create a fulfilling Star Schema. I work for a utility company, so some of the values or going to be MWH, Gj, m3 etc. I've made a description trying to describe my thoughts behind the grains and […]   [RSS Feed]([Twitter]( This email has been sent to {EMAIL}. To be removed from this list, please click [here](. If you have any problems leaving the list, please contact the webmaster@sqlservercentral.com. This newsletter was sent to you because you signed up at SQLServerCentral.com. ©2019 Redgate Software Ltd, Newnham House, Cambridge Business Park, Cambridge, CB4 0WZ, United Kingdom. All rights reserved. webmaster@sqlservercentral.com  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -