blog

I write about random stuff, mostly Kedro related. Some of them are my documentation when I was working through a PR.

 
How to achieve Partial Immutability with Python? dataclasses or attrs?
python
dataclasses
attrs
Should you use Python’s dataclasses or attrs? This article will give you an overview with examples.
noklam
Apr 22, 2022
 
(WIP) - Integrating Iceberg with Kedro for data versioning
python
kedro
Interestingly, the…
Nov 18, 2024
 
Achieve (Almost) dependencies-free Kedro Viz Pipeline
python
kedro
Kedro viz is a visualisation tools for Kedro. It creates an interactive flow chart that visualize Kedro’s pipeline in a web app. One of the issue is that…
Jul 8, 2024

diff v1

miniKedro - A minimal Kedro implementation
python
kedro
databricks
In this blog post, I am going to re-implement Kedro in 50 lines of code. I will implement the core components of Kedro such as ConfigLoader, DataCatalog in a…
May 31, 2024
 
How to save or load data using Kedro classes directly?
python
kedro
DataCatalog is the first concept that you learnt in Kedro. Although it is important, users actually never have to interact with it, at least not directly. This post will…
Mar 26, 2024

Select a node on kedro-viz

Kedro Pipeline (1) - Slicing Pipeline Effortlessly 🍕
python
kedro
databricks
Kedro Pipeline provides options allow you to slice and compose pipelines effortlessly
Mar 6, 2024

bundle folder structure

Kedro with Databricks Assets Bundle
python
kedro
databricks
Using Kedro with Databricks Assets Bundle
Feb 21, 2024

Enhancing Debugging experience with Jupyter Magic
python
kedro
jupyter
Jupyter Notebook has been thed default tooling for many. Despite many people trying to get rid of it, it integrates even more deeply with the data ecosystem and it is going…
Feb 20, 2024
 
Investigation of the Kedro default node names
python
kedro
https://github.com/kedro-org/kedro/issues/3575
Feb 8, 2024

Making Powerpoint Ready Chart with matplotlib
python
Are you tired…
Feb 8, 2024

How Kedro Config Loader works
python
kedro
Kedro Config Loader, resolution order
noklam
Nov 16, 2023
 
Advance Pytest - parameterize your fixture
pytest
Using pytest fixture with params combination
Nov 15, 2023
 
Experimenting Rust with Python
rust
Learnin Rust, one step at a time
Nov 11, 2023

A Guide to Kedro Namespace Pipelines for Time Series Forecasting
kedro
Learn how to use Kedro’s Namespace Pipelines for efficient time series forecasting.
Sep 26, 2023
 
Function overloading - singledispatch in Python with type hint
python
Using singledispatch with type hint
Nov 16, 2022
 
Mocking Python version for testing, use of Python Warning to raise error for specific Python Version
python
Mocking Python version for testing, use of Python Warning to raise error for specific Python Version
Nov 16, 2022
 
Kedro DuckDB
python
Kedro Meet the Duck
Nov 15, 2022
 
Understanding the Kedro codebase - A quick dirty meta-analysis - (Part I)
kedro
Meta-analysis of the kedro codebase
Nov 13, 2022
 
Being Python Expert
python
Notes for the talk - James Powell: So you want to be a Python expert? | PyData Seattle 2017
Nov 10, 2022
 
Demo of debugging Kedro pipeline with noetebook
python
kedro
Demo of debugging Kedro pipeline with noetebook
Nov 8, 2022
 
Quick implementation of Kedro DebugRunner
europython
Quick implementation of a debug runner
Nov 1, 2022

EuroPython 2022 - Conference Notes & Summary
europython
Notes for EuroPython2022 (Update daily)
Jul 11, 2022
 
Testing with Mocking
python
I have been working with on some unit tests recently with mocking. There are some traps that I falled into and I want to document it here.
noklam
May 30, 2022
 
Journey of understanding Python and programming language
python
Some unfinished notes about Python lower level details.
noklam
Feb 10, 2022

What can we learn from Shipping Crisis as a Data Scientist?
product
The Long Beach port congestion could teach us a lot about data science in real world.
noklam
Nov 18, 2021

Python FileNotFoundError or You have a really long file path?
python
Today I encountered an interesting bug that I think it is worth to write it down for my future self.
noklam
Aug 18, 2021
 
5 Minutes Data Science Design Patterns I - Callback
python
design pattern
software
A mini collections of design pattern for Data Science - Starting with callbacks.
noklam
Jul 10, 2021
 
Advance Kedro Series - Digging into Dataset Memory Management and CacheDataSet
python
kedro
Kedro pipeline offers some nice feature like automatically release data in memory that is no longer need. How is this possible? Let’s dive deep into the code.
noklam
Jul 2, 2021

Jupyter Superpower - Extend SQL analysis with Python
python
reviewnb
sql
Making collboration with Notebook possible and share perfect SQL analysis with Notebook.
noklam
Jun 26, 2021
 
A logging.config.dictConfig() issue in python
python
logging.config.dictConfig()
noklam
Jun 20, 2021
 
Python Internal Series - Global Interpreter Lock (GIL) and Memory Management
python-internal
Despite the bad reputation of GIL, it was arguably designed on purpose. The GIL actually comes with a lot of benefit.
noklam
May 29, 2021

RNN

Full Stack Deep Learning Notes - Lecture 03 - Recurrent Neural Network
fsdl
Lecture & Lab notes - This lecture is about Recurrent Neural Network. Key concetps included input gate, forget gate, cell state, and output gate. It also explains how…
noklam
Apr 16, 2021
 
Microsoft Azure - DP100
azure
This note helps you to prepare the Azure Assoicate Data Scientist DP-100 exam. I took DP100 in Mar 2021 and includes some important notes for study. Particularly, syntax…
noklam
Mar 27, 2021
 
Full Stack Deep Learning Notes - Lecture 01
fsdl
Lecture & Lab notes, explain DataModules, Trainer LightningModule.
Mar 21, 2021

deepcopy, LGBM and pickle
python
pickle
deepcopy
At first sight, these 3 things may not sounds related at all. I am writing this article to share a bug with lightgbm that I encountered and it eventually leads to deeper…
noklam
Mar 19, 2021
 
Data Test as CI
python
Unit Test with Pytest. I want to display a full information rich string into my CI Log, but it is trimmed by Python. Here is a simple fix to make it display full string in…
noklam
Mar 17, 2021
 
Setting up pyodbc for Impala connection, works on both Linux and Window
pyodbc
impala
Easiest way to connect with Impala in Windows
noklam
Mar 5, 2021

“Recreating the BBC style graphic in Python - plotnine and altair”
python
Do you know that you can have ggplot in BBC R graphics cookbook? This is an attempt to reproduce https://bbc.github.io/rcookbook/ in python
Apr 13, 2020

data augmentation - Understand MixUp and Beta Distribution
ML
Mixup and Beta Distribution
noklam
Feb 9, 2020
 
Hydra - Config Composition for Machine Learning Project
python
ML
GitHub: https://github.com/noklam/notadatascientist/tree/master/demo/hydra-example
noklam
Feb 8, 2020
 
plyer - Desktop Notification with Python
python
Desktop notifiaction with Python
noklam
Oct 19, 2019

nbdev + GitHub Codespaces: A New Literate Programming Environment
codespaces
nbdev
How a new GitHub feature makes literate programming easier than ever before.
Hamel Husain & Jeremy Howard
Jan 1, 2019
No matching items