472 subscribers
6 photos
1 video
2 files
550 links
python, go, code quality, security, magic

Website and RSS:
https://itgram.orsinium.dev

Source:
https://github.com/orsinium/itgram

Author:
@orsinium
https://orsinium.dev/
Download Telegram
πŸ“ Golang is evil on shitty networks is another reminder that networking is messy. The blog post and the related GitHub issue got some traction, and there are answers to it on the orange site from Russ Cox (the author of the code in question) and John Nagle (the author of the algorithm). It all together is a very interesting reading on networking issues and how it is solved in TCP.
Matching on negative literals results in a parser error. This Github issue is a good example of why I will never touch Elm. Elm is a very interesting functional language but gosh the governance is bad. The bug report is that Elm explodes if you try to use negative numbers in pattern matching, and this is the answer of Elm author:

> Why are you doing this? Instead of fake examples, can you explain how this comes up?

D-uh. There was a drama a long time ago that Elm banned using JS directly in third-party libraries (reserving this possibility for stdlib only), and that decision caused many people to leave the language community. I'm surprised anyone even stayed.

I guess the point of this post is that when picking a programming language, not only the language itself matters but also the community and the governance around it.

The Go core team is considering adding telemetry in Go toolchain (compiler, formatter, linter, language server, all that stuff) that is opt-in by default. I wonder how far this story would go, but I think it will be fine. There already were talks that Go is just another Google's product, they don't care about community, and generics (the initial proposal based on contracts) are getting pushed. And that story the Go team managed well. They canceled the controversial proposals, and what we now have in Go is much more community-accepted.

I could also rant about the Go community itself (and push-back on generics for years as an idea), but I should stop somewhere.
πŸ“ Functional Programming in C++ is a (mirror of) blog post from John Carmack about using functional approach in, well, any OOP language, actually, not just C++. I like how well-thought, practical, and balanced his vision on this. He covers pure functions and immutability, their trade offs, and how to fit into the messy OOP world. I think it's a good intro into the world of functional programming, just enough to get interested and already start using it at work.
TLA+ is a formal specification language to describe design of distributed systems and then let the computer find issues with that design. The name stands for "Temporal Logic of Actions". "Logic" means that you mostly work with boolean values and operations (you also have sequences, numbers, sets, and functions, though). "Temporal" means you can describe how your system changes over time (something will eventually happen or something always happens). And finally "Actions" means that your system is described as a state machine of states and transitions between them.

It is designed by Leslie Lamport (he's not the one who made the IDE and verifier, though), the creator of LaTeX. Hence the syntax of TLA+ is basically LaTeX. It's designed for mathematicians rather than programmers, and so describing imperative operation in it is mind-bending. So, Lamport later also created PlusCal, an imperative DSL that generates TLA+ code. TBH, it's all a big mess and far cry from modern programming languages, their syntax, and conveniences, but it gets the job done.

The best place to learn TLA+ and PlusCal is learntla.com by Hillel Wayne, his blog, and his Practical TLA+ book. His materials have the benefit of being designed for engineers, rather than any math-heavy and obscure-symbols-loaded materials from Lamport. BTW, the book is dirt cheap, it almost feels like Hillel pays half of the production price for you. He himself says that he made learntla.com because he didn't like that his book costs money.

Don't worry about any materials being ever outdated, TLA+ doesn't get any updates for 9 years.
So, the story goes like this: I learned TLA+ to my best, but it doesn't sparkle joy. I never wanted to program on LaTeX, or ever touch LaTeX for anything, really. And then I discovered Alloy.

Alloy is a formal verification language (and IDE for it) for describing categories of entities, relations between them, and constraints between these relations. For example you can have category User with relation follows to the set of users, and a user cannot follow themselves. That's how you describe it in Alloy:

sig User {
follows: set User
}

fact no_self_follows {
no u: User.follows {
u in u.follows
}
}


That's it! There are shorter ways to describe the constraint, but this way is more similar to what you would do in Python. And, as you can see, it feels like a real programming language, and the syntax is quite nice.

When you describe your system, you can check different constraints on it (do the formal verification) or simply press "show" in IDE, and it will produce different graphs of the system that fits your description (with one and many follows, DAGs, loops, and so on).

It all doesn't seem like much, but, thanks to Curry-Howard correspondence, you can describe using it anything: types, sets, database records, servers, boolean logic, etc.

Alloy never was considered as an alternative to TLA+ because it didn't have temporal logic. But things have changed just a few years with introduction in Alloy 6 of even more cool temporal operators than TLA+ has. So, now I see Alloy 6 as a good replacement for TLA+ for myself that is developer-friendly, readable, more powerful, and easier to wrap my head around. Alloy does sparkle joy!

Learning resources:

+ Finding bugs without running or even looking at code (or Alloy for TLA+ users, it's the same talk for different audience) is a good intro video.
+ Alloy Documentation is an unofficial documentation by Hillel Wayne about Alloy. At the moment of writing, it’s not updated for Alloy 6, so skip the "time" section. The rest is good.
+ As for TLA+, check out the tag Alloy in Hillel Wayne's blog, it has some good posts. I especially recommend Formally Modeling Database Migrations (doesn’t use time) and Alloy 6: it’s about time (covers the new time syntax).
πŸ“ Microfeatures I'd like to see in more languages. I don't find most of the listed features useful, but it's interesting to see little quirks of some not so famous languages.
Stop Writing Dead Programs is a great talk about some legacy features that modern programming languages keep (and so we keep using them) from the times of punchcards.

I'd a bit argue with the ending, though. I believe in visual programming languages too but I think it's quite a big leap there from how we program now. Some people go all-in for visual languages, some say to always bet on text. I think the trick is to find the balance there, and the next big language is something like Node-RED (with visual representation of high-level flow and text representation of logic inside of each function) rather than Enso where everything is visualized. I'd like to see better structure and navigation for modules and functions, as it was in smalltalk, and more interactive development, as it was in lisp. The ideas aren't new, we just need to get them back into modern languages. I'm currently deceiving a plan to get Livebook (Jupyter but better and for Elixir) to that level πŸ€”
Raft: Understandable Distributed Consensus is amazingly clear, beautiful, and interactive visualization of how Raft algorithm works. Raft is the algorithm for picking a leader node and synchronizing changes across multiple nodes in a distributed system. This is the magic behind the replication of RabbitMQ, Nats, Etcd, CockroachDB, and many others.
πŸ“ How do One-Time passwords work? A nice little blog post with an example on Go.
πŸ“ INTERCAL, YAML, And Other Horrible Programming Languages. TL;DR: we have lots of bad DSLs that are YAML with control structures attached on top of that.

With each year, fewer and fewer people laugh when I say "YAML developer". I think there are several reasons why we have YAML everywhere.

One is that when you just start a project, YAML seems like a reasonable configuration solution. It's relatively readable and there are parsers for all languages. And then one day you need control structures, and instead of rewriting everything, you simply add on top of it a template engine you like and call it a day.

Another reason is marketing. If you take a DevOps or JS engineer and introduce a new fancy tool, it's easier to persuade them to pick it up if it's YAML than if it's Python or Clojure. Everyone knows that programming languages are hard and YAML is easy, right?

I think DSLs are even worse than YAML, though. At least with YAML you already have syntax highlighting, a code formatter, and a basic linter. Out of all Makefile, I strongly prefer Taskfile because I can work with it without learning a yet another poorly made syntax and installing yet another glitchy vscode extension.

If you need a powerful configuration language with control structures, reusability, and predefined variables, go pick Starlark (I posted about it in Russian here and here). Or simply make a Python library instead of yet another DSL interpreter. Python is installed almost everywhere, many engineers know it good enough, and it's quite readable, especially for simple cases. With Python you get a huge ecosystem, linters, type checkers, code formatters, and all that cool tooling. For example, I made docked Python library that generates Dockerfile, and I alone could provide safety features and IDE support that Dockerfile cannot dream of.
πŸŽ₯ The Mess We're In is a talk by Joe Armstrong, a co-creator of Erlang, at Strange Loop conference. It's about complexity of modern system, about that chaos of duplicated and unreliable components that barely work.

The talk is 9 years old. Joe Armstrong died in 2019. Strange Loop conference will happen in this year last time. And yet the problems he covers are even more pronounced today and will stay true until the fall of humanity.
🐍 Pandas 2.0 is released! The biggest highlight is a support for pyarrow instead of numpy as a backend, which gives a better performance and a better support for missing values. A few months ago, there was a great post about it in the pandas blog: pandas 2.0 and the Arrow revolution.

The release notes are a bit too verbose, so I recommend reading this TL;DR instead: Everything you need to know about pandas 2.0.0.

#python
ruff is making a lot of buzz in Python community recently. It's a Python linter written in Rust. It re-implements all built-in rules from flake8, most of the rules from pylint, and a lot of flake8 plugins. On top of having it all out-of-the-box, it's very fast and provides autofix logic for many stylistic checks.

It's still in "beta", and a few months ago when I tried it for the first time it exploded on most of big projects I tried it on, but now it's stable enough. I recommend giving it a try on your pet projects.

For me, the number one feature of the tool is a vscode plugin that lints the code as you type it. It's something that I had in Atom years ago and what I've been missing the most in vscode. It even has autofixes as refactoring actions!

From the disadvantages, I'd mention that it brings the clumsy configuration approach from flake8 and still doesn't have a way to write plugins for it. But I think fixing it is just a matter of time.

BTW, did you know Atom had been archived in March? It was objectively less stable and consistent than vscode, but it had some nice features, and it's always great for ecosystem to have alternatives. I still use Atom hotkeys and theme in vscode. Probably, since Microsoft acquired Github, ditching Atom in favor of vscode was inevitable.

#python
msgspec is a serialization and validation library for Python with schemas defined declaratively with type annotations (like dataclasses). Think of it as a pydantic alternative. And it has quite a few advantages:

1. Performance. In their benchmarks they claim to tear apart everything else, especially pydantic. And while Pydantic 2.0 with the core implemented on Rust is on horizon, it's not clear when it will be finished, and I still expect msgspec to stay ahead.

2. No implicit type conversion. If the filed type is specified as int and you pass in a string, pydantic (by default) will try its best to convert the passed value to the int. While it is useful for some cases, like parsing URL or GET query parameters, most often you'll use it for interacting with other APIs and providing your own API, and there types are better to be strict.

3. Clean and consistent API. Pydantic has a relatively long history, and it has accumulated a fair share of bad decisions. Again, it's something that the author wants to fix in pydantic 2.0, which isn't clear when it will come out. Msgspec has a clean and functional API that doesn't mess with your classes.

4. Out-of-the-box support for many formats: JSON, YAML, TOML, and MessagePack. And if you need more, there is a function that converts your data into JSON-compatible primitive types that you can then serialize as you want.

A nice bonus is the support for dataclass and TypedDict, so you can give it a try without rewriting the models you have.

#python
πŸ“ How Software Companies Die. How is it written in 28 years ago and describes all the places I worked in? In 1995, I was busy non-existing. I don't think it's how companies die, though. More like how software products turn into unmaintainable pile of garbage. And garbage products can sell just fine. Take Jira for example.
πŸ“ An aperiodic monotile exists. A big math discovery the beauty of which in its simplicity. The article explains it very well. TL;DR: Mathematicians have found a shape that you can use to tile with an infinitely big floor without gaps and without repetitions of the pattern. I think I know what I lasercut next.
An Animated Introduction to Elixir. This is a guide into Elixir with quite an interesting format. Each chapter is build around a code example (similar to Go by example) and when you press the "play" button, you see how the code changes a bit, and on the left there is an explanation for the change, with links and images. Beautiful! It only works on desktop, though.

#elixir
πŸ“ Type embedding: Golang's fake inheritance is a great beginner-friendly blog post about inheritance in Go. It shows how it's different from OOP, what's good and bad about it, how it plays with interfaces, and how (not) to shoot yourself in a foot.

#golang
πŸ¦€ The built-in Rust testing framework is alright for small tests, but it quickly falls apart when the project grows. And even for small projects, the tests look messy and the failure output isn't helpful. I haven't found one all-in framework like pytest but there are some small projects that together make something much better:

+ nextest is a fast test runner with a nice colorful output, powerful DSL for filtering tests to be run, and built-in mechanism for detecting slow and flaky tests. And the great thing is that it runs regular Rust tests, so you don't need to change anything in the prject itself to use it.

+ proptest is a hypothesis-inspired framework for writing property-based tests. You tell it what kind of input you want for your test, and it generates random values, trying to cover as many corner cases as possible. When a test fails, it finds a nice and small example for reproducing the failure.

+ rstest is a library that provides pytest-like fixtures and test parametrization (for table-driven testing). I'm not sure if fixtures is a good idea (I wrote pytypest for Python just to fix the mess that pytest fixtures bring in big projects) but parametrization is a must. You can also try test-case library, it looks similar but also lets you to specify a name for each case.

+ k9 provides a macro for snapshot testing. It dumps on disk whatever data you pass into it on the first run, and later on consequent runs compares the new value with the dumped one.

+ A better assert statement is still in question. The built-in assert! and assert_eq! do not provide a helpful enoug error message, far cry from what you can see in pytest or ex_unit. k9 and pretty-assertions provide some additional assertions but that's still not enough. In particular, I right away miss something like assert_is_close for comparing floats or an assertion that will show the arguments with which the target function was called. There is approx library just for that but I think adding a library for each assertion type doesn't scale well.

I wonder if the built-in Rust macro system allows making a nice looking assert macro with pytest-like source rewrites. Like a macro that will convert magic_assert!(f(a, b) > x) into assert!(f(a, b) > x, "f({}, {}) > {}", a, b, x).

#rust
πŸ¦€ Found it! assert2 solves the problem I described above. You simply write assert!(a + b > 13) and if it fails, instead of "something wrong, good luck" it will say "5 + 6 > 13", with nice colors and all. Now my basic set of human-friendly testing for Rust is complete.

#rust
πŸ“ I investigated the Underground Economy of Glassdoor Reviews is an unexpectedly detailed post about how companies buy online reviews on Glassdoor and similar platforms. The author contacted freelancers providing such services, aggregated their quotes and services, asked some "hows", and built interesting charts for a few random companies. The research perhaps lacks scale, but still, quite an interesting rabbit hole.