Tuning the tools you create: testing code highlighting in IDEs

Explore key techniques for testing code highlighting in IDEs to ensure your tools work across different platforms [View this email in your browser]( by Tamás Balog | [Read online at Ministry of Testing]( Testers are most likely familiar with the testing pyramid: unit and component tests, various levels of integration tests, and the rest. Tools or applications, be they open or closed source, for customer or internal use, all have their own specific rules on how to use them correctly and optimally. It’s beneficial for teams to include automatic validations like static code analysis checks in these tools. Static code analysis checks allow a program to be tested without actually executing the code. This is not to be confused with syntax highlighting which is highlighting the keywords and elements a programming language offers. Techniques like static code analysis help teams to ensure that the tools they build can be used as intended, especially when those tools will be used widely in the industry. Such checks can help uncover coding issues early in the development process, and can even help engineers learn how to operate those tools. Static code analysis capability is especially important for developers of code editors and IDEs (Integrated Development Environments), as well as for IDE plugin developers. This is because those who create programming languages, test frameworks, and so forth must also implement static code analysis to make it available to the engineers who will use the tools. However, it is not enough to implement those checks. It is also crucial to test them to ensure that they work properly, so that they don’t mislead or confuse users by falsely reporting correct code as incorrect (or vice versa). A little background Before I started to develop JetBrains IDE plugins, I worked as a test automation engineer. I developed unit tests for our test automation frameworks, Selenium-based system integration tests, and web UI tests. When I was introduced to testing IDE plugins, at first it seemed straightforward to use unit testing and object mocking for certain IDE features. It was not an overly complex thing to do, but after a while I started to realize that it is overkill. When all you have is a hammer, everything looks like a nail. I found out that there are more convenient ways of performing those tests. See each section below for more information on optimal ways to perform each test. 👋 Prompting for Testers —Unleash the power of generative AI to boost your software testing and day-to-day tech tasks On-demand course is launching on Monday! Different IDE platforms, different testing capabilities In the next sections, I’ll show you how I tested code highlighting in the editors of three of the major IDEs for which I developed plugins. Each IDE offers different capabilities and approaches to testing its features, so to simplify things, I’ll move from a less than ideal solution, a unit testing approach, to a more fitting one, a sort of visual testing approach on the integration level. Although my experience is limited in terms of Visual Studio and VS Code extension development (I develop plugins mostly for JetBrains IDEs), there may be better solutions than the ones in this article. However, they still demonstrate well what I outlined in the introduction. Please note that the sections below contain code snippets from three different languages, namely C#, TypeScript, and Java, so a bit of technical knowledge is needed. I have tried to keep the snippets as simple and lean as possible. With all that said, let me start with an example. An example problem Let's say we have a simple method with a single vararg parameter into which an even number of name-value pairs must be passed. C# example: void SetHeaders(params string nameValuePairs) { ... } We want to report and underline the method name when an odd number of arguments are passed in (for example, a header name doesn’t have a value specified). A call to this method that should trigger reporting could be something like the following. anObject.SetHeaders("header", "value", "another header"); Now, let’s see an example of what I implemented in a Visual Studio extension. [Read full article...]( 👋 MoT Software Testing Essentials Certificate — a modern introduction to the world of software testing. [Early access pricing now available]( [Website]( [LinkedIn]( [YouTube]( [Twitter]( [Instagram]( Copyright © 2024 Ministry of Testing, All rights reserved. You have opted to join this email list. Our mailing address is: Ministry of Testing 19 New RoadBrighton, East Sussex BN1 1UF United Kingdom [Add us to your address book]( Want to change how you receive these emails? You can [update your preferences]( or [unsubscribe from this list](.