Converting Huge CSV Files to Parquet with Dask, DuckDB, Polars, Pandas, PyArrow
Converting Huge CSV Files to Parquet with Dask, DuckDB, Polars, Pandas, PyArrow
PyPI has completed its first external security audit, 6 Types of Python Dictionaries, Data Class in Python, Python: Hidden Features — part 1 and It's Time For A Change: datetime.utcnow()
PyPI has completed its first external security audit
It’s a three-part series. The audit was focused on "Warehouse", the open-source codebase that powers https://pypi.org, and on "cabotage", the custom open-source container orchestration framework we use to deploy Warehouse. The good part is there were no high-severity issues in either codebase.
# Dustin Ingram
Converting Huge CSV Files to Parquet with Dask, DuckDB, Polars, Pandas, PyArrow
Compared Pandas, Polars, PyArrow, Dask and DuckDB and explained well and mainly author gave closure instead of being diplomatic. When the 8 GB file was handled Polars did a great job on every angle, DuckDB was not the fastest library but it had low memory consumption.
# Mariusz Kujawski
6 Types of Python Dictionaries: How to Choose the Right One
In this Author explained about 6 types of Python dictionaries in the standard library - dict, collections.defaultdict, collections.OrderedDict, collections.ChainMap, collections.Counter and collections.UserDict. Several times I counted the frequency of elements in a sentence or paragraph for that wrote code by myself after reading this understood collections.Counter easily, It seems I wasted my time and energy in past :)
# Fateme Fouladkar
Data Class in Python
You might heard of the “DataClass” word several times maybe you might have a good idea about it if not then a data class is a class that is designed to only hold data values. This article did a pretty good job of explaining Data Class with good examples.
# Anand Rathore
Python: Hidden Features — part 1
When I read its title I thought another clickbait but I read several articles of Pravash so went ahead and read it. As usual, he did a great job. In this article, he explained mutability issues with function objects, how to make class objects behave like a function, “startswith” and “endswith” hidden feature, frozenset usage and Pickle Files
# Pravash
It's Time For A Change: datetime.utcnow() Is Now Deprecated
datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version. In this article, he gives a bit of context and alternatives for them using datetime.now(), datetime.fromtimestamp() with timezone.utc