Clik here to view.
The low-power mode for STM8L and ATTINY2313
I used to think STM8L would cost less energy than ATTINY. Today I got some time to view the datasheet for these two MCU. First comes the STM8L datasheet: If we just use low speed oscillator (LSI) in...
View ArticleClik here to view.
First experiments about Vertex AI of Google Cloud
As the above menu show in the Vertex AI, it is trying to include all common processes of building and running a machine learning model. For my experiment, I just create a Dataset by loading file from...
View ArticleThe “real size” of hex file for AVR microcontroller
Recently I dug out my USBasp tool and a few AVR microcontrollers, for enjoying programming the C language again. Unexpectedly, the old ATTINY2313V and ATmega88V couldn’t work with my USBasp tool...
View ArticleHow to paste in Vim
After I have written the SQL on BigQuery’s UI: SELECT first_column FROM `project.dataset.table` ORDER BY `first_column` ASC LIMIT 1000 I tried to copy it to my Vim editor through “Ctrl + c” and “Ctrl...
View ArticleSome tips in groovy of Jenkins
Groovy could be used as the configuration file for Jenkins workflow. Although I totally don’t make head or tail of Groovy, its syntax is not hard to learn. How to iterate a list To export a bunch of...
View ArticleKubeflow deployment: part 1
By following the document, I tried to deploy the management cluster of Kubeflow. But after running make apply-cluster it reported: The management cluster name "kubeflow-mgmt" is valid. # Delete the...
View ArticleClik here to view.
My summary for the paper “Unified Language Model Pre-training for Natural...
For NLU (Natural Language Understanding), we use the bidirectional language model (like BERT), but for NLG(Natural Language Generation), the left-to-right unidirectional language model (like GPT) is...
View ArticleKubeflow deployment: part 2
Instead of using other strange components of Kubeflow, we can just use Kubeflow Pipelines which is much easier to be deployed and used. After deployment (less than 20 minutes), I test my first example...
View ArticleSome hints on Dataproc
When running a job in the cluster of Dataproc, it reported: java.util.concurrent.ExecutionException: java.lang.ClassNotFoundException: Failed to find data source: BIGQUERY. The reason is I haven’t...
View ArticleClik here to view.
Kubeflow deployment: part 3
After upgrading my k8s cluster, all the jobs of Kubeflow Pipelines will only finish the first operation and hang there. The reason is a bug in Argo (Kubeflow is based on Argo). And the most simple and...
View ArticleHow to mount multiple Kubernetes secrets into one directory
As the title, Kubernetes already has a new component called `projected volume` that support the mounting of multiple secrets into one directory. apiVersion: v1 kind: Pod metadata: labels: run: my-app...
View ArticleStrange permission problem
I tried to run a basic example of the Google Vertex AI pipeline: import kfp from kfp.v2 import compiler from kfp.v2.google.client import AIPlatformClient from google.cloud import aiplatform from...
View ArticleEncrypt and decrypt a string in shell
When using a container in Kubernetes, I prefer to use a shell command to complete the task instead of writing a python script since the shell commands are usually much simpler and intuitive. If I want...
View ArticleGet DDL of a table in BigQuery
How could I conveniently get the creating-SQL of a table in BigQuery? We could use INFORMATION_SCHEMA: SELECT table_name, ddl FROM `data-to-insights.taxi.INFORMATION_SCHEMA.TABLES` WHERE...
View ArticleClik here to view.
Python code for the n-queens problem (using NumPy)
I am reading the second edition of “The Algorithm Design Manual” recently. Reaching the chapter about backtracking, I realized that this method could solve many problems, even complicated problems,...
View ArticlePython code for the sequence partition problem (using NumPy)
Imagine we have an array of numbers [9, 8, 7, 1, 2, 3, 4, 5, 6]. What’s the best solution to split it into 3 partitions with the most “average sum”, which means they have minimum differences for their...
View ArticleFinding problem about ‘Nan’ result in model training
Intending to use distilling for training my model. The Plan is: Train model A and model B with same code and same datasetPredict the dataset with model A and model B, and store the average of their...
View ArticleGet the number of rows for a parquet file
We were using Pandas to get the number of rows for a parquet file: import pandas as pd df = pd.read_parquet("my.parquet") print(df.shape[0]) This is easy but will cost a lot of time and memory when...
View ArticleA handy Python library
To find an easy way to print the runtime of a function, I found an interesting Python library: funcy. We can directly use decorator ‘@print_durations()’ like this: from funcy import print_durations...
View ArticleHow to get the number of CPU cores inside a container
We usually use the below Python code to get CPU cores: from multiprocessing import cpu_count print("CPU cores:", cpu_count()) But when the snippet running inside a docker container, it will just...
View Article