Stairway to Database Containers Level 4: Using a Docker Compose File (2024-08-28)

SQLServerCentral Newsletter for August 28, 2024 Problems displaying this newsletter? [View online](. [SQL Server Central]( Featured Contents - [Stairway to Database Containers Level 4: Using a Docker Compose File]( - [Microsoft Fabric Warehouse Security: Object-Level, Column-Level and Row-Level]( - [Is your DevOps strategy missing a vital link?]( - [From the SQL Server Central Blogs - Migrate Your Azure Data Studio Connection List]( - [From the SQL Server Central Blogs - Observing Extended Events]( - [The Phoenix Project]( Question of the Day - [Converting Integers]( The Voice of the DBA  A Kafka Introduction I've heard of Kafka before. I know it's an Apache project and you can download or read more at [(. I knew it was a way of moving data around, some sort of ETL tool useful for moving things around. More like a message and queueing system, which is a tool that seems like a great idea, but one that everyone struggles to work with. And one that seemed complex. The overview is that Kafka is "a distributed system consisting of servers and clients that communicate via a high-performance [TCP network protocol](. It can be deployed on bare-metal hardware, virtual machines, and containers in on-premise as well as cloud environments." Would I need that or use it? In a lot of my database work, I'm not sure that it would easily fit into most of the OLTP applications or data warehouse systems. Maybe. Hard to tell. Their description of event streaming and the definition of an event make it seem this is a catch-all system for moving log data around. One that be so open-ended that it ends up requiring a lot of configuration for "my" system. Here's their definition of an event: An event records the fact that "something happened" in the world or in your business. It is also called record or message in the documentation. When you read or write data to Kafka, you do this in the form of events. Conceptually, an event has a key, value, timestamp, and optional metadata headers. Recently I watched a [Kafka presentation]( at THAT Conference (which was a fantastic event). In the talk, this sentence caught my eye: "[Kafka is] a pipe to move data from A to B, C, D". I've certainly had that need, and sometimes configuring lots of pipes is work. If you've ever worked with replication and the publisher/subscriber model you likely get a twitch in your eye if a ticket is opened to configure a new subscriber. Not because the configuration is hard, but because the ongoing admin can be a pain. The talk dives into some of the complexity of designing and implementing a Kafka system. For developers that might write to the stream or read from it, things seem simple. For admins and architects, less so, and I can't help what happens when a reader goes down. I have nightmares of replication subscribers being down and transaction logs not being reused. Kafka doesn't seem as complex as I thought before, but it certainly doesn't seem simple or easy. Kafka is not a panacea for moving data around, but it is a well-understood and widely used technology. Those things mean more to me now that I find myself considering the challenges of maintaining a system over time and hiring staff who understand it. It's something I'd consider using in the future, and maybe something I'd like to experiment with a bit more and learn how it works at a more practical level. If you use it, or know more, I'd be interested in how well Kafka has worked for you, either as a developer or admin. Steve Jones - SSC Editor [Join the debate, and respond to today's editorial on the forums](   Featured Contents [Container Stairway icon]( [Stairway to Database Containers Level 4: Using a Docker Compose File]( Steve Jones - SSC Editor from SQLServerCentral In the next level of the Stairway to Database Containers, let's learn to use a compose file to specify a number of options for our container. [External Article]( [Microsoft Fabric Warehouse Security: Object-Level, Column-Level and Row-Level]( Additional Articles from MSSQLTips.com Learn about object-level, column-level, and row-level security in Microsoft Fabric Warehouse and how this can be implemented to limit access to data. [External Article]( [Is your DevOps strategy missing a vital link?]( Additional Articles from Redgate Database DevOps practices can help you ship features faster, but how do you safely enable this, and ensure those updates will successfully deploy? Redgate Test Data Manager was developed to resolve bottlenecks and challenges, helping organizations ship deployments they can trust by enabling developers and testers to self-serve test data that walks and talks like production with sensitive customer data taken care of. Read more on the blog. [Blog Post]( From the SQL Server Central Blogs - [Migrate Your Azure Data Studio Connection List]( hellosqlkitty from SQLKitty I wanted to migrate some connections without recreating them and wondered how to do that in Azure Data Studio. It turns out to be easy. I post about whatever... [Blog Post]( From the SQL Server Central Blogs - [Observing Extended Events]( Grant Fritchey from The Scary DBA Sure, you can right click on a running session for Extended Events and open the Live Data window, but are there other ways of observing what Extended Events is... [The Phoenix Project]( [The Phoenix Project]( Additional Articles from SQLServerCentral In this newly updated and expanded edition of the bestselling The Phoenix Project, co-author Gene Kim includes a new afterword and a deeper delve into the Three Ways as described in The DevOps Handbook.   Question of the Day Today's question (by Steve Jones - SSC Editor):  Converting Integers The max integer value is 2,147,483,647. What is returned from this code? SELECT 2147483649 / 2 AS Result2; Think you know the answer? [Click here]( and find out if you are right.    Yesterday's Question of the Day (by Steve Jones - SSC Editor) The SQL Server 2022 32 Bit Upgrade I have a SQL Server 2014 SP4 instance that is a 32 bit version of SQL Server. Can I upgrade this to SQL Server 2022 with the setup program? Answer: No, cross-platform upgrades are not supported Explanation: There is no checkbox in setup. Cross-platform upgrades are not supported, and this includes 32-bit to 64-bit. Ref: Unsupported Scenarios - [ [Discuss this question and answer on the forums](    Database Pros Who Need Your Help Here's a few of the new posts today on the forums. To see more, [visit the forums](. --------------------------------------------------------------- SQL Server 2017 - Administration [Does a Query Lock to the Table_Name or Table_ID]( - SQL 2017 ENT SCENARIO: Table_1 has a Sch-M lock on during an intended updating process. A QUERY is executed against Table_1 and is queued up, waiting for the Sch-M lock to release. While the QUERY is queued up on Table_1, Table_1 is renamed to Table_1_OLD A newly designed Table_2 is then renamed to Table_1 QUESTION: […] SQL Server 2016 - Development and T-SQL [Help with to generate Data pattern Sql help]( - Need to generate a from and to From to 50 -99 100 -299 300 -- I tried to do the logic ,but its working as 1-50 51-100 101 -  FinalCTE AS ( Select ,FROMQUANTITY AS S_FROMQUANTITY ,ROW_NUMBER() OVER(PARTITION BY C.po_line_id ORDER BY C.FROMQUANTITY ASC) AS A ,CASE WHEN CAST(C.FROMQUANTITY AS NUMERIC(18,7)) >= 1.0 THEN […] [Losing Saved Connections in SSMS 2016]( - Microsoft SQL Server Management Studio 13.0.16106.4 Hi , I am used to having saved connections in the Connect to Server popup. For some reason my SQL Server management studio 2016 is not saving them anymore. Please help me get saved connections back or get the connections to save again.  SQL Server 2019 - Administration [SQL Server Availability Group Database Backup Missing Data]( - I have a production SQL2019 Enterprise SQL Server which hosts an Availability Group with 12 small to medium databases in it. This is an OLTP server and there is no database over 200gb. The AG only has 2 nodes and is used for H/A when patching. Both nodes are onPrem so the Availability Mode is […] SQL Server 2019 - Development [ssis dashboard suddenly missing from reports option under catalog/ssisdb]( - hi i rt clicked ssisdb today under integration services catalog , chose reports and didnt see the dashboard option under my regular id. my id has sa. so i connected with my admin which also has sa and did see it. losing permissions would have occurred in the last 24 hrs. i saw in one […] [checking on some restrictions related to functions]( - hi i'm helping a friend on his creation of a function.  here are the issues... he's getting an error setting nocount on. i told him that is only a proc thing as far as i know.  is that correct? 2. he's getting an error setting xact abort on.  i was a little surprised by this. […] SQL Azure - Development [Hybrid Cloud Server Integration with SQLServerCentral’s SQL Server 2022]( - Hello everyone, I’m currently exploring the use of hybrid cloud servers and am focusing on integrating them with SQLServerCentral’s SQL Server 2022. While hybrid cloud environments offer flexibility, managing and optimizing them with SQL Server 2022 presents several challenges. Here are some specific areas where I’m seeking advice: Integration Best Practices: What are the best […] Powershell [dump to csv -- image column]( - If there is an Image Column in mssql table and I wanted to use export-csv to dump this info to file how do you handle that column to be able to import that csv to another DB. Thanks. SQL Server 2022 - Administration [Alias woes on SQL Server 2022]( - Our company is moving to the cloud. I have SQL Developer Edition 2022 on Windows Server 2022 Datacenter setting up our QA SharePoint Farm.. I've created a named instance running over port 55055 using TCP/IP. THis is what I have in my on premises SQL server that we set up about 10 years ago. The […] [Db level user is not having execute permission no db_owner]( - Dear all, we have a strange situation Db level [ user ]is not having execute permission also not having db_owner , having ddl read and write but in audit logs it is showing the same user had action_id/EX permission in audit logs how can this is possible. [SQL Server 2022 Memory Consumption]( - I have migrated to SQL Server 2022 from SQL Server 2014 and after migration I have observed that it is consuming more memory than before. Memory is going almost 80-90%. We have same configuration, there is not memory caping on SQL Server 2022 and there was no caping on SQL Server 2014. We have same […] SQL Server 2022 - Development [SSIS package project connections don't work on upgrade]( - We're upgrading from SQL Server 2016 to 2022. The SSIS package project level connections don't work as expected. One queries the server as an anonymous logon instead of the domain account associated with the SQL Server Agent. One even affects both the original server and the new one to which it is pointed. I have […] [2022 azure vm cant run same ssis pkg we run on prem]( - Hi, we set up an environment on our new azure 2022 vm with a cloned "prod target warehouse", ssis project clone from prod, cloned sql agent etc etc. amongst other things the ssis points at a new erp's sales data we are testing with in just one agent step/ssis project execution for that 2019 std […] [ssis 32 bit vs 64, debug and non debug]( - Hi i am looking in vs2022 at my imported 2022 dev azure vm based ssis project. And vs2022 at my 2019 non azure non vm (i think) based production ssis project. The idea is to compare their bittages after i got an error on the dev box basically saying the 64 bit connector wasnt installed […] [Strange(?) performance problems]( - I am debugging an NHibernate generated SQL_query for performance problem and came up with something strange. The query is a SELECT statement that reads from a view. I've tracked the problem to this part of the query: AND (myView.A LIKE (@p7+'%') OR myView.B=@p8 OR myView.C=@p9) If I run the query with only condition at a […]   [RSS Feed]( This email has been sent to {EMAIL}. To be removed from this list, please click [here](. If you have any problems leaving the list, please contact the webmaster@sqlservercentral.com. This newsletter was sent to you because you signed up at SQLServerCentral.com. ©2019 Redgate Software Ltd, Newnham House, Cambridge Business Park, Cambridge, CB4 0WZ, United Kingdom. All rights reserved. webmaster@sqlservercentral.com  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -