noclip.website is "a digital museum of video game levels". It contains levels from different games from different platforms rendered online with WebGL. Very interesting. For instance, now I know that the mirror floor in "Hotel Ocean View" from GTA Vice City is, in fact, a half-transparent floor containing a mirrored copy of the room below. Mirrors in most of videogames is a lie.
In Elixir community, there are a lot of talks about PETAL stack for dynamic websites. It stands for Phoenix, Elixir, Tailwind CSS, Alpine.js, and LiveView. The order of components doesn't make sense (except making it sound cool), so I'll cover them out of order:
+ Elixir is a functional programming language that has out-of-the-box scalability, hot reload, and other cool features that each big project needs.
+ Phoenix for elixir is like Ruby on Rails for Ruby. The default web framework. Big, powerful, and popular. Even if you write a very simple and small Elixir web app, you're likely to still use Phoenix for some of the conveniences inside.
+ LiveView is now part of Phoenix. And this is a huge game changer in web development. If you know PHP, it's similar to LiveWire. It allows you to have dynamic web pages where all content is rendered on the server side. LiveView will take care of sending data in both directions through websocket and make sure everything is stable and accounts for tons of corner-cases. At the end of the day, you will have a modern reactive website without writing a line of JS.
+ Alpine.js is for cases when you still need a bit of JS for some frontend-only logic. It's a micro framework. With LiveView, you don't really need React or anything huge like this because LiveView provides you with all nice logic, state management, components, and all that stuff. And alpine.js covers the rest. I used it for filters on my personal website and it works well. Well, it eats too much resources which is noticeable on a phone but it was easy and fun to code. So, I guess, worth it.
+ Tailwind CSS is a controversial one. It's basically CSS in your classes. When I need something low-level, I use CSS. When I need something high-level which takes care of nice defaults and adaptivity, I use bootstrap. And when I mix bootstrap with some custom CSS, I get all the same things that Tailwind CSS promises but with less boilerplate. IDK, I just don't understand yet what Tailwind tries to solve.
Someone even made petal_components Elixir library which contains Phoenix and LiveView components on top of Tailwind CSS and Alpine.js.
#elixir #web #js
+ Elixir is a functional programming language that has out-of-the-box scalability, hot reload, and other cool features that each big project needs.
+ Phoenix for elixir is like Ruby on Rails for Ruby. The default web framework. Big, powerful, and popular. Even if you write a very simple and small Elixir web app, you're likely to still use Phoenix for some of the conveniences inside.
+ LiveView is now part of Phoenix. And this is a huge game changer in web development. If you know PHP, it's similar to LiveWire. It allows you to have dynamic web pages where all content is rendered on the server side. LiveView will take care of sending data in both directions through websocket and make sure everything is stable and accounts for tons of corner-cases. At the end of the day, you will have a modern reactive website without writing a line of JS.
+ Alpine.js is for cases when you still need a bit of JS for some frontend-only logic. It's a micro framework. With LiveView, you don't really need React or anything huge like this because LiveView provides you with all nice logic, state management, components, and all that stuff. And alpine.js covers the rest. I used it for filters on my personal website and it works well. Well, it eats too much resources which is noticeable on a phone but it was easy and fun to code. So, I guess, worth it.
+ Tailwind CSS is a controversial one. It's basically CSS in your classes. When I need something low-level, I use CSS. When I need something high-level which takes care of nice defaults and adaptivity, I use bootstrap. And when I mix bootstrap with some custom CSS, I get all the same things that Tailwind CSS promises but with less boilerplate. IDK, I just don't understand yet what Tailwind tries to solve.
Someone even made petal_components Elixir library which contains Phoenix and LiveView components on top of Tailwind CSS and Alpine.js.
#elixir #web #js
verifier is a package that makes your Go code a bit nicer when you need to validate multiple things before running a block of code.
#golang
verify := verifier.New()They don't say it, but this is, basically, a monad. The package exists because Go doesn't have exceptions or powerful enough type system 👀 Anyway, the library is pretty helpful, remember it when you next time need to validate a lot of things.
verify.That(transfer != nil, "transfer can't be nil")
verify.That(person != nil, "person can't be nil")
if verify.GetError() != nil {
return nil, verify.GetError()
}
#golang
Drunk Post: Things I've learned as a Sr Engineer. That's better than you might expect from the title.
Not IT but Gram: @laser_gram is my new small channel with photos of things I cut out of wood on my laser cutter. I also have @rpgdump with some random findings about TTRPGs and board games, and @pythonetc about Python which I hope get back to soon.
I'm surprised I haven't published it before. 🎥 Type-Driven Development in Idris is a great talk from Edwin Brady, the creator of Idris language, about, well, Idris.
Idris is a language with the best type system out there. Brady has other newer talks about Idris 2, a new more powerful version of Idris, so from the point of view of semantics this talk is a bit outdated. It was published 7 years ago! But I still think this is the best one because I want you to learn not Idris but the fundamental ideas behind it.
The Idris type system has a few very powerful features that your language, most likely, doesn't:
+ Higher-kinded types: you can write functions that operate with types as values.
+ Refined types: you can use very specific types, like "non-empty list", "even integer", "positive number", "hexadeciaml string", etc.
+ Dependent types: you can call functions from type annotations, so some types will be generated depending on types and values of other arguments. A classic example is
I watched this talk about 3 years ago when type annotations in Python were quite new thing for me and for the most of the Python community. And it changed how I think about types and static analysis in general. Showed, how cool and powerful it all might be. It inspired me to bring more static analysis into deal, including a basic formal verification.
There is my advice. Doesn't matter which language you write on, run all static analysis you can. It's almost free. Writing type annotations for everything is no-brainer, you know what type your function expects and returns anyway. And more you write annotations, cleaner they will be, because you start to think about your code in a more structured, "type-safe" manner. And good linters, even if they have false-positives, it happens not so often and usually indicates that your code is over-complicated and too magical anyway. I happened to meet multiple times engineers that were "smarter than linters", and no, they were not.
#idris
Idris is a language with the best type system out there. Brady has other newer talks about Idris 2, a new more powerful version of Idris, so from the point of view of semantics this talk is a bit outdated. It was published 7 years ago! But I still think this is the best one because I want you to learn not Idris but the fundamental ideas behind it.
The Idris type system has a few very powerful features that your language, most likely, doesn't:
+ Higher-kinded types: you can write functions that operate with types as values.
+ Refined types: you can use very specific types, like "non-empty list", "even integer", "positive number", "hexadeciaml string", etc.
+ Dependent types: you can call functions from type annotations, so some types will be generated depending on types and values of other arguments. A classic example is
printf function in C-like languages (or str.format in Python) where the types and amount of arguments depend on the template you pass as the first argument. For instance, if you pass "%s has %d messages", the function needs exactly 2 more arguments, and the first one must have type String and the second one must have type Integer. If you pass wrong arguments, most of the languages (I know for sure it's true for C and Go) will fail at runtime, unless you use a linter. In Idris, the error will be correctly detected at compilation time.I watched this talk about 3 years ago when type annotations in Python were quite new thing for me and for the most of the Python community. And it changed how I think about types and static analysis in general. Showed, how cool and powerful it all might be. It inspired me to bring more static analysis into deal, including a basic formal verification.
There is my advice. Doesn't matter which language you write on, run all static analysis you can. It's almost free. Writing type annotations for everything is no-brainer, you know what type your function expects and returns anyway. And more you write annotations, cleaner they will be, because you start to think about your code in a more structured, "type-safe" manner. And good linters, even if they have false-positives, it happens not so often and usually indicates that your code is over-complicated and too magical anyway. I happened to meet multiple times engineers that were "smarter than linters", and no, they were not.
#idris
If you Google "data visualization in Python", most of the tutorials you'll see will mostly cover matplotlib. The truth is that, yes, matplotlib is very powerful and flexible tool but at the same time very low-level and hard to use. So, plenty of wrappers emerged on top of it.
The most famous wrapper is seaborn. It is very high-level and has sensible defaults to make nice looking commonly used charts with a call to a single function.
My personal favorite is plotnine, I use it all the time when I need to visualize something. It's based on the idea of "the grammar of graphics" introduced in R library ggplot2. The idea is pretty simple. In ggplot2 (and so in plotnine) the visualization consists of multiple layers: data (a data frame), aesthetics (what rows and aggregations to use for ox, oy, color, etc.), and one or more geometries (a way to visualize data using the aesthetic, like boxplot, bars, etc.). There are also "facets", "scales", and "stats", but they are also just layers.
Why this is cool:
1. It's easy to learn, explain, and understand.
2. It's easy to use and it's not so much code.
3. It's flexible, you can make any visualizations and combine them in any manner you want.
This is why the recent alpha release of seaborn (v0.12.0a1) introduced Next-generation seaborn interface. The article nods to ggplot2 and plotnine as the sources of inspiration for the new interface. But it also says the interface will be different and more "Pythonic". All that sounds very cool. Maybe, the new seaborn will win my heart over plotnine ❤️
Both libraries are one-man projects. That's crazy how much time some people can invest in a single project without getting paid. I was there but I gave up at some point on dephell when I just couldn't sleep at night for hours because of anxiety about unresolved issues... Be mindful of your health, folks.
#python
The most famous wrapper is seaborn. It is very high-level and has sensible defaults to make nice looking commonly used charts with a call to a single function.
My personal favorite is plotnine, I use it all the time when I need to visualize something. It's based on the idea of "the grammar of graphics" introduced in R library ggplot2. The idea is pretty simple. In ggplot2 (and so in plotnine) the visualization consists of multiple layers: data (a data frame), aesthetics (what rows and aggregations to use for ox, oy, color, etc.), and one or more geometries (a way to visualize data using the aesthetic, like boxplot, bars, etc.). There are also "facets", "scales", and "stats", but they are also just layers.
Why this is cool:
1. It's easy to learn, explain, and understand.
2. It's easy to use and it's not so much code.
3. It's flexible, you can make any visualizations and combine them in any manner you want.
This is why the recent alpha release of seaborn (v0.12.0a1) introduced Next-generation seaborn interface. The article nods to ggplot2 and plotnine as the sources of inspiration for the new interface. But it also says the interface will be different and more "Pythonic". All that sounds very cool. Maybe, the new seaborn will win my heart over plotnine ❤️
Both libraries are one-man projects. That's crazy how much time some people can invest in a single project without getting paid. I was there but I gave up at some point on dephell when I just couldn't sleep at night for hours because of anxiety about unresolved issues... Be mindful of your health, folks.
#python
LISP in Space is a cool episode of CoRecursive podcast about automating spacecrafts with LISP, when the automation was just making its way into the field. I won't spoil too much, just listen to it.
I listened a few more episodes of the show. I also can recommend The Original Remote Developer, a story of the guy who made the first graphical text editor for Apple leaving alone in a cabin in a forest.
I listened a few more episodes of the show. I also can recommend The Original Remote Developer, a story of the guy who made the first graphical text editor for Apple leaving alone in a cabin in a forest.
📚Software Engineering at Google is a free book from Google about engineering practices: how to lead, test, deprecate, document, measure, enforce, and review. I don't really read that kind of books, too many unnecessary words, but this one has TL;DRs. I've read all of them, and some chunks of the text that looked interesting, new, and relevant.
Some random highlights:
> there are usually a few things that everyone would like to do in the next five years: be promoted, learn something new, launch something important, and work with smart people.
> What's the best way to raise awareness about testing in a company with employees scattered around the world? After a little bit of brainstorming, someone proposed the idea of posting flyers in the restroom stalls as a joke. We quickly recognized the genius in it: the bathroom is one place that everyone must visit at least once each day, no matter what. Joke or not, the idea was cheap enough to implement that it had to be tried.
> To change the quality of engineering documentation, engineers need to accept that they are both the problem and the solution.
Some random highlights:
> there are usually a few things that everyone would like to do in the next five years: be promoted, learn something new, launch something important, and work with smart people.
> What's the best way to raise awareness about testing in a company with employees scattered around the world? After a little bit of brainstorming, someone proposed the idea of posting flyers in the restroom stalls as a joke. We quickly recognized the genius in it: the bathroom is one place that everyone must visit at least once each day, no matter what. Joke or not, the idea was cheap enough to implement that it had to be tried.
> To change the quality of engineering documentation, engineers need to accept that they are both the problem and the solution.
🔧 scc is a very fast and simple CLI tool for calculating lines of code in a codebase. It supports multiple languages and additionally implements an interesting stat about how many engineers, months, an money is needed to re-implement the project from ground-zero. What's interesting, I've tested it on some of our internal projects, and the result is surprisingly accurate, something like ±5%. The big difference from most of other similar tools is the support for modern languages (like Go) and
.gitignore. Oh, and performance is amazing.🔧directus is a powerful and user-friendly WebUI panel for a database. It connects to any SQL database, and provides UI for creating tables and fields, viewing and managing records, webhooks, REST API, 2FA, audit logs, versioning, roles, import and export, custom dashboards, and a lot of other stuff. The best thing you can get if you need a friendly, almost-zero-config, admin panel.
There is also nocodb but it feels more like a spreadsheet rather than an admin panel. The biggest feature they are proud of (and what makes it better for some cases than Directus) is a bunch of integrations for automating workflow. It also has some quite specific representations for data, like kanban, calendar, file view, gallery. Depending on what you need, also worth checking out.
There is also nocodb but it feels more like a spreadsheet rather than an admin panel. The biggest feature they are proud of (and what makes it better for some cases than Directus) is a bunch of integrations for automating workflow. It also has some quite specific representations for data, like kanban, calendar, file view, gallery. Depending on what you need, also worth checking out.
If you need to make a new REST API, please, don't reinvent the wheel. Take JSON:API. This is a good enough standard for writing REST APIs, and using a standard gives a lot of benefits:
1. Ready-to-use libraries for both client and server sides.
2. Well-documented. Just point your users to there, and you have less stuff to write.
3. Easy to achieve consistency in API of different endpoints.
4. Well-thought design decisions made by smart people based on their pain and experience.
Also, consider using GraphQL. The choice between GraphQL and a REST API depends on if you want to have more control on the server or on the client side. GraphQL allows frontend team go faster and always easily request just the data they need. REST API allows to write a well-thought, fast, and optimized backend.
1. Ready-to-use libraries for both client and server sides.
2. Well-documented. Just point your users to there, and you have less stuff to write.
3. Easy to achieve consistency in API of different endpoints.
4. Well-thought design decisions made by smart people based on their pain and experience.
Also, consider using GraphQL. The choice between GraphQL and a REST API depends on if you want to have more control on the server or on the client side. GraphQL allows frontend team go faster and always easily request just the data they need. REST API allows to write a well-thought, fast, and optimized backend.
You might've received recently a few spam messages from diffgram. I didn't pay much attention to it until I stumbled across this discussion: Open source tool scrapes git commit email addresses to send spam to. And they are removing all issues with complaints. Especially noteworthy is this PR: Stop spamming. It clearly shows the malicious actions of the maintainers.
That's why we can't have nice things. I used to have my main email in my GitHub profile, git commits, and metadata of Python packages. This story motivated me to create separate emails for each of these places (I have limit in 10 emails in my current plan, so I can't have a new one everywhere). And contact GitHub support about this story.
I think I can relate to this story. Well, they are doing business, so sending spam is financially viable. But even if it wouldn't be the case, there is a real desperation when you put a lot of effort into a project, but it gets drown in the sea of billions other low quality projects. There is no correlation between GitHub stars and usefulness of the project. Take as an example nocode which has 53k stars but close to zero effort and zero usefulness. Just a little joke from a famous guy. So, if you feel your effort isn't appreciated and doesn't get attention it deserves, you're not alone. Don't be desperate, don't overthink it, and don't go spamming.
That's why we can't have nice things. I used to have my main email in my GitHub profile, git commits, and metadata of Python packages. This story motivated me to create separate emails for each of these places (I have limit in 10 emails in my current plan, so I can't have a new one everywhere). And contact GitHub support about this story.
I think I can relate to this story. Well, they are doing business, so sending spam is financially viable. But even if it wouldn't be the case, there is a real desperation when you put a lot of effort into a project, but it gets drown in the sea of billions other low quality projects. There is no correlation between GitHub stars and usefulness of the project. Take as an example nocode which has 53k stars but close to zero effort and zero usefulness. Just a little joke from a famous guy. So, if you feel your effort isn't appreciated and doesn't get attention it deserves, you're not alone. Don't be desperate, don't overthink it, and don't go spamming.
ITGram
You might've received recently a few spam messages from diffgram. I didn't pay much attention to it until I stumbled across this discussion: Open source tool scrapes git commit email addresses to send spam to. And they are removing all issues with complaints.…
If you're interested, the response from GitHub on my report.
🏃Go 1.19 is released. Well, it was released at the very beginning of August, so I'm a bit late with the news. Well, very late. Still, I wanted to tell you about it. This release of Go is notable in that it doesn't have anything really notable, like any other Go release. Since Go 1.5 or so, most of the releases had mostly performance and quality of life improvements. The only notable exceptions are error wrapping in Go 1.13 (which is a small change, but it made error handling suck less, which is great) and generics in Go 1.18. That means, all old Go code still works, most of the old tutorials are still as good as they always were, and upgrades are safe and smooth.
It's also great that often all you need to do to make your code faster is to upgrade to the latest Go version. This is why the article from Discord about migrating from Go to Rust got so much negative feedback. They had performance issues on go 1.6 when 1.12 already was around, which is 2-3 times faster. Just try to upgrade before rewriting everything, common!
Alright-alright, there is one worth mentioning feature in Go 1.19. It's not a change in the language itself but rather one more knob to tweak GC. Now you can set a soft memory limit for your app, so you want accidentally eat all RAM and die. Read more here: GOMEMLIMIT is a game changer for high-memory applications.
It's also great that often all you need to do to make your code faster is to upgrade to the latest Go version. This is why the article from Discord about migrating from Go to Rust got so much negative feedback. They had performance issues on go 1.6 when 1.12 already was around, which is 2-3 times faster. Just try to upgrade before rewriting everything, common!
Alright-alright, there is one worth mentioning feature in Go 1.19. It's not a change in the language itself but rather one more knob to tweak GC. Now you can set a soft memory limit for your app, so you want accidentally eat all RAM and die. Read more here: GOMEMLIMIT is a game changer for high-memory applications.
🐍 debuglater is a Python library that saves stacktrace into a file when your app crashes with an exception. Not just a traceback, but the whole stack, including all variables. It's all is saved using dill, so if you have the same code, you can load the saved stacktrace into a pdb session and debug the crash, as if you could if you had put an actual breakpoint into the place of failure. Pretty cool, huh?
I just found this introduction to heredoc in Dockerfiles, and it's pretty cool. It won't allow you to do anything that you couldn't do in Dockerfiles before, but it makes the syntax much better and easier to work with. The main idea is that you won't need to add
&& \ at the end of each command inside of RUN. Give it a try.STEALING UR FEELINGS is an interactive movie about privacy and machine learning that shows you exactly how ML can be used to steal your privacy. I highly recommend you show it to your non-IT friends and family members. Don't worry, though, it doesn't steal anything itself (I hope), it's open-source.
Today I learned that you can import a CSV file into SQLite and then use it to run SQL queries on the data you have. I often feel that it would be easier to just run a short SQL query than try to craft some monstrosity with pandas. Now I know exactly how. Also, in the original issue someone posted a bash function for convenience:
# Usage: cq <csv_file> [<table>]
cq() {
local csv_file="${1:?Usage: cq <csv_file> [<table>]}" table="${2:-T}"
sqlite3 -interactive -cmd '.headers on' -cmd '.prompt "\nsqlite> "' \
-cmd '.mode csv' -cmd ".import '$csv_file' '$table'" -cmd ".mode column"
}