Getting resulting statuses of CI jobs is of course significant. For instance, action-shellcheck is a github action to show violations of shellcheck. Reviewdog is an automated code review tool integrated with any code analysis tools regardless of programming language. $ sqlfluff lint models/ = FAIL L: 1 | P: 1 | L003 | Single indentation uses a number of spaces not a multiple of 4 L: 1 | P: 14 | L006 | Operators should be surrounded by a single space unless at the start/end of a line L: 1 | P: 27 | L001 | Unnecessary trailing whitespace What is reviewdog? We can also fine-tune rules in a configuration file. As it is very flexible, we can easily customize coding standard by selecting rules from the reference. For instance, sqlfluff lint enables us to find violations of SQL coding style. SQLFluff has some sub commands for a SQL linter and auto-formatter. There are some other linter for SQL, but quire few target dbt. The reason why I use sqlfluff is that it is the best open-sourced linter for dbt at the moment. So, I am assuming many dbt users take advantage of SQLFluff on their projects so that they detect violations of SQL queries. A strong point for dbt users is that it officially supports JINJA template and dbt models. It supports various dialects not only ANCI SQL but also BigQuery, Hive, Redshift, Snowflake and so on. SQLFluff also works with Jinja templating and dbt. It is a dialect-flexible and configurable SQL linter. SQLFluff is a SQL linter and auto-formatter. By automating reviewing style check, we can be more focus our time on what matters. In this article, I would like to describe automating SQL coding style leveraging sqlfluff which is a SQL linter and auto-formatter and reviewdog which leaves comments about violations of coding style to GitHub pull requests. Standardized coding style on SQL for dbt can help us to maintain even complicated SQL queries. As I am a dbt lover, I would like to automatically check coding style for team development taking advantage of CI, especially GitHub Actions. Especially, integrating static code analysis tools with continuous integration (CI) is very useful. It makes our programming code more readable and maintainable as well. One of the most basic measures is static code analysis used to flag programming errors, bugs, stylistic errors and suspicious constructs. Today, we are bringing many DevOps concepts to the data engineering world.
0 Comments
Leave a Reply. |