Capture the return status of a shell command
Let’s start by looking at the code of the shell: ls whatever if [ $? -eq 0 ]; then echo "success" else echo "fail" fi Since there is no file named ‘whatever’ in the current directory, the return code...
View ArticleClik here to view.
A wrong way to solve
About two weeks ago I met the “number of islands” problem in LeetCode. The first idea that jumped out of my brain is ‘dynamic programming’: I can create a new matrix (let’s name it ‘number matrix’)...
View ArticleNaN value in DataFrame
NaN value in NumPy or Pandas is some type dangerous for data processing. Like the example below: import numpy as np import pandas as pd df = pd.DataFrame({"col1": [None, None, None]}) df["col1"] =...
View ArticleClik here to view.
Try to deploy Argo on GCP Autopilot
Creating an Autopilot cluster in GCP K8S is quite easy. But after deploying Argo and launching our pipeline, the Argo report errors: Failed to pull image...
View Articlepandas.datetime64 with Timezone
I barely pay attention to the pandas.datetime64 type. But yesterday a problem stroke me. It was a parquet file with a column “start_date”: >>> df["start_date"] start_date 0 2022-03-22...
View ArticleRunning Django in docker
I am trying to learn Django (a python framework for developing web) in docker container recently. After running the docker with port redirect sudo docker run -p 8000 -t -i centos/django /bin/bash #in...
View ArticleClik here to view.
Using ARM64 on Oracle Cloud
Because of my wife’s strong recommendation, I started to use Oracle Cloud to host my blog, for free Currently, the Oracle Cloud provides four 4-core/24GB ARM64 virtual machines without any fees. This...
View ArticleClik here to view.
Books I read in the year 2021
I know it looks too late that I wrote this article in the middle of the year. But, late is much better than none. Right? I bought the seven books series of “A song of ice and fire” on 23rd November...
View ArticleA problem with “gcloud auth login”
When I am trying to log in to my account in a Google Engine VM with “gcloud auth login“, it jumps out a hint: You are authorizing gcloud CLI without access to a web browser. Please run the following...
View ArticleModel saving error when using Apex
Apex is a tool to enable mixed-precision training that comes from Nvidia. import apex.amp as amp net, optimizer = amp.initialize(net, optimizer, opt_level="O2") # forward outputs = net(inputs) loss =...
View ArticleSome test samples for Text-To-Speech solutions
I am doing some research on TTS (Text-To-Speech) recently and noticed three almost state-of-the-art and also out-of-the-box solutions: LightSpeech (from Microsoft), FastSpeech2 (partly from...
View ArticleDownload files from Google Drive in the console
I need to download some large files from Google Drive on my server (“server” means no GUI). After a quick search, I got a solution: https://stackoverflow.com/a/50670037/5048046 We can just install it...
View ArticleClik here to view.
Using PyTorch on ClearLinux docker image
I am using Nvidia’s official docker image of PyTorch for my model training for quite a long time. It works very well but the only problem is that the image is too large: more than 6GB. In my poor home...
View ArticleInsert multiple lines in a specific position of a file
I used awk for quite a long time, but not his brother sed. A couple of days ago I want to insert two lines for a CMake file in a specific position and find a perfect answer: here. Now I could add two...
View ArticleFix “unsupported-assignment-operation” error for Pandas
When using pylint to check my code, it reported an error: E1137: 'df' does not support item assignment (unsupported-assignment-operation) for the origin code: df["column1"] = "hello" It looks like we...
View ArticleAverage weights of two Pytorch models
After reading this paper, I begin to do an experiment about it. Referencing this snippet, I wrote my code: net1 = model_builder.build_model() net2 = model_builder.build_model() output =...
View ArticleAccelerate inference speed of DNN on Intel CPU
To save the cost on the inference server, I did some experiments on how to accelerate the speed of prediction for our model. import torch.nn as nn import pycls.core.builders as model_builder from...
View ArticleAn interesting problem about ext4 mounting
When I login my computer and try to run “tmux attach” this morning, it reported a strange error: /tmp/tmux-1001/default (Address already in use) Intuitively, I thought this temporary file is out of...
View ArticleThe correct way to insert data from another table in BigQuery
Incorrect code: WITH source1 as ( SELECT blah FROM blah ), source2 as ( SELECT moreblah FROM source1 ) INSERT INTO newtable FROM source2; Correct solution: INSERT INTO newtable WITH source1 as (...
View ArticleConditional checking in Argo workflow
My company has been using Argo for executing workflow for more than three years. I knew every step in the Argo workflow could be controlled by when expression, like this: apiVersion:...
View Article