If you need to make a new REST API, please, don't reinvent the wheel. Take JSON:API. This is a good enough standard for writing REST APIs, and using a standard gives a lot of benefits:
1. Ready-to-use libraries for both client and server sides.
2. Well-documented. Just point your users to there, and you have less stuff to write.
3. Easy to achieve consistency in API of different endpoints.
4. Well-thought design decisions made by smart people based on their pain and experience.
Also, consider using GraphQL. The choice between GraphQL and a REST API depends on if you want to have more control on the server or on the client side. GraphQL allows frontend team go faster and always easily request just the data they need. REST API allows to write a well-thought, fast, and optimized backend.
1. Ready-to-use libraries for both client and server sides.
2. Well-documented. Just point your users to there, and you have less stuff to write.
3. Easy to achieve consistency in API of different endpoints.
4. Well-thought design decisions made by smart people based on their pain and experience.
Also, consider using GraphQL. The choice between GraphQL and a REST API depends on if you want to have more control on the server or on the client side. GraphQL allows frontend team go faster and always easily request just the data they need. REST API allows to write a well-thought, fast, and optimized backend.
You might've received recently a few spam messages from diffgram. I didn't pay much attention to it until I stumbled across this discussion: Open source tool scrapes git commit email addresses to send spam to. And they are removing all issues with complaints. Especially noteworthy is this PR: Stop spamming. It clearly shows the malicious actions of the maintainers.
That's why we can't have nice things. I used to have my main email in my GitHub profile, git commits, and metadata of Python packages. This story motivated me to create separate emails for each of these places (I have limit in 10 emails in my current plan, so I can't have a new one everywhere). And contact GitHub support about this story.
I think I can relate to this story. Well, they are doing business, so sending spam is financially viable. But even if it wouldn't be the case, there is a real desperation when you put a lot of effort into a project, but it gets drown in the sea of billions other low quality projects. There is no correlation between GitHub stars and usefulness of the project. Take as an example nocode which has 53k stars but close to zero effort and zero usefulness. Just a little joke from a famous guy. So, if you feel your effort isn't appreciated and doesn't get attention it deserves, you're not alone. Don't be desperate, don't overthink it, and don't go spamming.
That's why we can't have nice things. I used to have my main email in my GitHub profile, git commits, and metadata of Python packages. This story motivated me to create separate emails for each of these places (I have limit in 10 emails in my current plan, so I can't have a new one everywhere). And contact GitHub support about this story.
I think I can relate to this story. Well, they are doing business, so sending spam is financially viable. But even if it wouldn't be the case, there is a real desperation when you put a lot of effort into a project, but it gets drown in the sea of billions other low quality projects. There is no correlation between GitHub stars and usefulness of the project. Take as an example nocode which has 53k stars but close to zero effort and zero usefulness. Just a little joke from a famous guy. So, if you feel your effort isn't appreciated and doesn't get attention it deserves, you're not alone. Don't be desperate, don't overthink it, and don't go spamming.
ITGram
You might've received recently a few spam messages from diffgram. I didn't pay much attention to it until I stumbled across this discussion: Open source tool scrapes git commit email addresses to send spam to. And they are removing all issues with complaints.…
If you're interested, the response from GitHub on my report.
🏃Go 1.19 is released. Well, it was released at the very beginning of August, so I'm a bit late with the news. Well, very late. Still, I wanted to tell you about it. This release of Go is notable in that it doesn't have anything really notable, like any other Go release. Since Go 1.5 or so, most of the releases had mostly performance and quality of life improvements. The only notable exceptions are error wrapping in Go 1.13 (which is a small change, but it made error handling suck less, which is great) and generics in Go 1.18. That means, all old Go code still works, most of the old tutorials are still as good as they always were, and upgrades are safe and smooth.
It's also great that often all you need to do to make your code faster is to upgrade to the latest Go version. This is why the article from Discord about migrating from Go to Rust got so much negative feedback. They had performance issues on go 1.6 when 1.12 already was around, which is 2-3 times faster. Just try to upgrade before rewriting everything, common!
Alright-alright, there is one worth mentioning feature in Go 1.19. It's not a change in the language itself but rather one more knob to tweak GC. Now you can set a soft memory limit for your app, so you want accidentally eat all RAM and die. Read more here: GOMEMLIMIT is a game changer for high-memory applications.
It's also great that often all you need to do to make your code faster is to upgrade to the latest Go version. This is why the article from Discord about migrating from Go to Rust got so much negative feedback. They had performance issues on go 1.6 when 1.12 already was around, which is 2-3 times faster. Just try to upgrade before rewriting everything, common!
Alright-alright, there is one worth mentioning feature in Go 1.19. It's not a change in the language itself but rather one more knob to tweak GC. Now you can set a soft memory limit for your app, so you want accidentally eat all RAM and die. Read more here: GOMEMLIMIT is a game changer for high-memory applications.
🐍 debuglater is a Python library that saves stacktrace into a file when your app crashes with an exception. Not just a traceback, but the whole stack, including all variables. It's all is saved using dill, so if you have the same code, you can load the saved stacktrace into a pdb session and debug the crash, as if you could if you had put an actual breakpoint into the place of failure. Pretty cool, huh?
I just found this introduction to heredoc in Dockerfiles, and it's pretty cool. It won't allow you to do anything that you couldn't do in Dockerfiles before, but it makes the syntax much better and easier to work with. The main idea is that you won't need to add
&& \ at the end of each command inside of RUN. Give it a try.STEALING UR FEELINGS is an interactive movie about privacy and machine learning that shows you exactly how ML can be used to steal your privacy. I highly recommend you show it to your non-IT friends and family members. Don't worry, though, it doesn't steal anything itself (I hope), it's open-source.
Today I learned that you can import a CSV file into SQLite and then use it to run SQL queries on the data you have. I often feel that it would be easier to just run a short SQL query than try to craft some monstrosity with pandas. Now I know exactly how. Also, in the original issue someone posted a bash function for convenience:
# Usage: cq <csv_file> [<table>]
cq() {
local csv_file="${1:?Usage: cq <csv_file> [<table>]}" table="${2:-T}"
sqlite3 -interactive -cmd '.headers on' -cmd '.prompt "\nsqlite> "' \
-cmd '.mode csv' -cmd ".import '$csv_file' '$table'" -cmd ".mode column"
}
🏃 maths is a math package for Go, with generics. And the readme deserves a special mention. It is beautiful, informative, well structured, and actively uses mathjax for markdown recently introduced on Github.
ziglings, rustlings, and gopherlings are tutorials for learning Zig, Rust and Go respectively by fixing bugs in tiny broken programs. It's great when you need some practice in a new language but don't have an idea or time for a proper project.
microservices.io describes different patterns and tricks for building microservice architecture. Because everyone nowadays wants some microservices. Well, it's a great way to decouple things when you have a big project and lot of people, but they introduce a lot of tech challenges and are hard to do correctly. I hope this guide will help you a bit if you have to go that way.
Testing in 2021 is a collection of good thoughts from one guy on testing. Very good thoughts. Without extremes and fanaticism, which is rare nowadays, especially in mainstream articles. Everyone says something "the only working testing is TDD" or "pyramid of testing is complete garbage". Stop making a religion from it. We need to get better at testing than we are now but think with your head, don't make dogmas. Oh, and the article also deserves a bonus point for having an audio version.
Today I want to tell you a bit about Livebook. It's like Jupyter Lab but for Elixir. And it is a bit of an understatement because Livebook is so much better. There are many reasons for it: Elixir is concurrent, they built the tool based on the negative and positive experiences of Jupyter, they already had a great Phoenix web framework to build it, and the team behind it are the go authors themselves. They fixed a lot of Jupyter issues, including out-of-order cell execution, custom widgets, and nice markdown rendering.
Livebook is a part of a bigger initiative of Elixir authors to bring Elixir into the world of ML, and they are doing great. There is nx for tenors, explorer for dataframe, axon for neural networks, and many other promising projects. And I think functional programming fits ML and datascience just great, much better than the messy world of OOP, mutations, and global states.
Livebook is a part of a bigger initiative of Elixir authors to bring Elixir into the world of ML, and they are doing great. There is nx for tenors, explorer for dataframe, axon for neural networks, and many other promising projects. And I think functional programming fits ML and datascience just great, much better than the messy world of OOP, mutations, and global states.
How we got to LiveView is a long read from Chris McCord, the creator of the Phoenix web framework for Elixir. It covers his experience with Ruby on Rails, what issues it had, how he fixed it in Phoenix, and then covers how LiveView is a continuation of these ideas.
Yes, I'm telling a lot here about Elixir, and I know that nobody cares. Still, I hope one day when you start a highly-concurrent and soft-realtime project with enough freedom to choose the tech stack, you may remember all my stories about Elixir, and give it a try. It's a relatively young language, but it is built on a very solid foundation and many great ideas.
Yes, I'm telling a lot here about Elixir, and I know that nobody cares. Still, I hope one day when you start a highly-concurrent and soft-realtime project with enough freedom to choose the tech stack, you may remember all my stories about Elixir, and give it a try. It's a relatively young language, but it is built on a very solid foundation and many great ideas.
If you want to learn Rust, the official website provides you 3 options: book, course, and examples. I started with the examples, and they are very bad. The first example is a classic Hello World but instead of just showing how to print text and move on, the resource goes into a lengthy explanation of what traits you need to inherit in your custom classes to make them printable. Without even explaining what is trait. And the same trend goes through all examples: dive into some details and use a bunch of terms without explaining them or even linking. I'm still not sure who's the target audience.
So, if you want to learn Rust, read the book, the book is pretty good.
The language itself is pretty strange too. Most modern languages realized that code is for humans, and the central goal of syntax and most of the high-level features should be readability and the principle of least surprise. And then there is Rust which consists of some obscure symbols, implicit behaviors, and strange decisions.
Rust requires each line of code to end with a semicolon. This is a strange decision for a modern language on its own. Most modern languages realized that you can tokenize code without it, and explicit semicolons are just visual noise.
But the best example of strange decisions is how to return a result from a function in Rust. In Python and Go, you always have to write an explicit return statement. In Elixir, Haskell, and Ruby function implicitly return the last expression. So, if you don't want to return anything, you make sure that the last expression is nil, which is, again, explicit.
Now, rust solves it in a very unique way. Instead of having an explicit return statement or nil at the end, it decides whether to return the expression or not based on if there is a semicolon at the end. If the line doesn't end with a semicolon, the result is returned. If it has a semicolon, a void is returned. Now, imagine you're reading some Rust code without going through the whole Rust book. What are the odds you'll figure it out? Some people will point out that the compiler will fail with a detailed error message if the semicolon and type annotation don’t match, but I’m talking about the experience of reading the code, not writing it.
At some point, people stopped using concise but hard-to-read languages, like Perl or APL. Well, there are still there but definitely not mainstream. Also, languages started to get rid of visual noise. You'll hardly find semicolons in most of them. I can't recall modern languages (except for PHP) that require a special symbol each time you need a variable. Many strictly typed languages, like Go, have some basic type inference. And here we are again, Rust gets into the mainstream with compiler-centric syntax.
So, if you want to learn Rust, read the book, the book is pretty good.
The language itself is pretty strange too. Most modern languages realized that code is for humans, and the central goal of syntax and most of the high-level features should be readability and the principle of least surprise. And then there is Rust which consists of some obscure symbols, implicit behaviors, and strange decisions.
Rust requires each line of code to end with a semicolon. This is a strange decision for a modern language on its own. Most modern languages realized that you can tokenize code without it, and explicit semicolons are just visual noise.
But the best example of strange decisions is how to return a result from a function in Rust. In Python and Go, you always have to write an explicit return statement. In Elixir, Haskell, and Ruby function implicitly return the last expression. So, if you don't want to return anything, you make sure that the last expression is nil, which is, again, explicit.
Now, rust solves it in a very unique way. Instead of having an explicit return statement or nil at the end, it decides whether to return the expression or not based on if there is a semicolon at the end. If the line doesn't end with a semicolon, the result is returned. If it has a semicolon, a void is returned. Now, imagine you're reading some Rust code without going through the whole Rust book. What are the odds you'll figure it out? Some people will point out that the compiler will fail with a detailed error message if the semicolon and type annotation don’t match, but I’m talking about the experience of reading the code, not writing it.
At some point, people stopped using concise but hard-to-read languages, like Perl or APL. Well, there are still there but definitely not mainstream. Also, languages started to get rid of visual noise. You'll hardly find semicolons in most of them. I can't recall modern languages (except for PHP) that require a special symbol each time you need a variable. Many strictly typed languages, like Go, have some basic type inference. And here we are again, Rust gets into the mainstream with compiler-centric syntax.
🎥 Seven Ineffective Coding Habits of Many Programmers by Kevlin Henney (text summary). I'm surprised I haven't published it before. A great conference talk about how humans read code, and so how to make your code a bit easier to read and understand.
This talk is what inspired me to create flake8-length that keeps line length short without compromising readability of messages, and flake8-comments that tries to catch useless comments.
This talk is what inspired me to create flake8-length that keeps line length short without compromising readability of messages, and flake8-comments that tries to catch useless comments.
🐍 I've made svg.orsinium.dev. It's an online playground for svg.py, and it has quite a potential. You can generate images in your browser. It's like processing but it's Python! Unfortunately, I there is no autocomplete in it (yet?), and a nice autocomplete is the whole point of svg.py. Still, quite useful for debugging your art.
The svg.py turned out to be more useful for me than I expected, TBH. I've created it to draw cards for my board game, and I still didn't get to it, shame on me. But despite of this, I already have used svg.py in punchline for producing punch stripes for my music box, and in generative-art which has only 4 images so far, but I hope it to grow.
The svg.py turned out to be more useful for me than I expected, TBH. I've created it to draw cards for my board game, and I still didn't get to it, shame on me. But despite of this, I already have used svg.py in punchline for producing punch stripes for my music box, and in generative-art which has only 4 images so far, but I hope it to grow.
🔧 netlify is quite cool. It's a hosting for static websites with a lot of nice features, like custom domains, automatic integration with Github (provides CI and previews), automatically renewed let's encrypt certificates, configuration-as-code. I use it for all my open-source websites (wps.orsinium.dev, orsinium.dev, svg.orsinium.dev, gweb.orsinium.dev, you got the idea), and I'm quite far from running out of quotas they provide for free accounts.
I avoided for quite a long time all static website hostings because I thought they only can serve whatever HTML you already have in the repo. No, netlify can also build the pages. For each build, it spawns an Ubuntu Docker image with a lot of stuff already installed (like Python and Go) and calls whatever custom command you've provided (which can be a shell script if you have a complex build logic). For example, in svg-playground, the netlify.toml tells to run netlify.sh which installs and runs task. And task executes Taskfile.yaml which does quite a few things, producing at the end all HTML pages and WASM binary. That could be fewer files if you want but I felt like being more verbose there.
I avoided for quite a long time all static website hostings because I thought they only can serve whatever HTML you already have in the repo. No, netlify can also build the pages. For each build, it spawns an Ubuntu Docker image with a lot of stuff already installed (like Python and Go) and calls whatever custom command you've provided (which can be a shell script if you have a complex build logic). For example, in svg-playground, the netlify.toml tells to run netlify.sh which installs and runs task. And task executes Taskfile.yaml which does quite a few things, producing at the end all HTML pages and WASM binary. That could be fewer files if you want but I felt like being more verbose there.
Registration for Hacktoberfest 2022 is open. Hacktoberfest is yearly event when you get some swag (T-shirt+stickers, or donation for planting a tree) for submitting 4 PRs to open-source projects during October.
Hacktoberfest 2020 had a lot of spam, so this year the initiative got some improvements, even more than in 2021:
1. Maintainers also will receive swag for reviewing and triaging Hacktoberfest PRs.
2. If 2 of your PRs are marked as spam, you automatically blocked.
3. You have to press 4 different checkboxes that you acknowledge the previous point.
Hacktoberfest 2020 had a lot of spam, so this year the initiative got some improvements, even more than in 2021:
1. Maintainers also will receive swag for reviewing and triaging Hacktoberfest PRs.
2. If 2 of your PRs are marked as spam, you automatically blocked.
3. You have to press 4 different checkboxes that you acknowledge the previous point.
SimpleParsing is a little Python library that "does one job and does it well".
If you, like me, want everything in your Python code to be type annotated (for the sake of autocomplete, semantic syntax highlighting, and safety) you may know that the container that argparse returns (argparse.Namespace) isn't type checked because mypy can't statically know what flags and of what type you registered in ArgumentParser. And if you want to make it type-safe, there is quite a bit of boilerplate: define flags, define type safe container, unpack flags into the container. And if the definition and the container mismtch, well, your code is wrong.
SimpleParsing solves exactly this problem. You define a dataclass class, annotate attribute types, set defaults, add comments to attributes, and then SimpleParsing will turn it into CLI args. Attribute names will form names of the CLI flags, types will be checked, defaults will be respected, and comments will be turned into help messages.
I've seen quite a few alternatives for that task, and all of them work on top of pydantic, click, or attrs. And I really like that SimpleParsing works with what we already have in stdlib, without bringing unnecessary dependencies.
And don't listen to anyone, click sucks. Testing it is quite hard, functions with 20 arguments and 20 decorators on top aren't nice, and passing all that stuff deeper into the code is hard and verbose. Using click will encourage dirty and verbose code and bad practice. Oh, and IMHO the CLI it produces is worse than that of argparse.
#python
If you, like me, want everything in your Python code to be type annotated (for the sake of autocomplete, semantic syntax highlighting, and safety) you may know that the container that argparse returns (argparse.Namespace) isn't type checked because mypy can't statically know what flags and of what type you registered in ArgumentParser. And if you want to make it type-safe, there is quite a bit of boilerplate: define flags, define type safe container, unpack flags into the container. And if the definition and the container mismtch, well, your code is wrong.
SimpleParsing solves exactly this problem. You define a dataclass class, annotate attribute types, set defaults, add comments to attributes, and then SimpleParsing will turn it into CLI args. Attribute names will form names of the CLI flags, types will be checked, defaults will be respected, and comments will be turned into help messages.
I've seen quite a few alternatives for that task, and all of them work on top of pydantic, click, or attrs. And I really like that SimpleParsing works with what we already have in stdlib, without bringing unnecessary dependencies.
And don't listen to anyone, click sucks. Testing it is quite hard, functions with 20 arguments and 20 decorators on top aren't nice, and passing all that stuff deeper into the code is hard and verbose. Using click will encourage dirty and verbose code and bad practice. Oh, and IMHO the CLI it produces is worse than that of argparse.
#python