Some tips about using Keras
1. How to use part of a modelmodel = load_model(sys.argv[1], custom_objects = {'fmeasure': fmeasure}) branch_model = model.get_layer(name = 'model_1') img_embed = Model(inputs =...
View ArticleUsing ResNeXt in Keras 2.2.4
To use ResNeXt50, I wrote my code as the API documentation for Keras:keras.applications.resnext.ResNeXt50(...)But it reported errors:AttributeError: module 'keras.applications' has no attribute...
View ArticleSome summaries for Kaggle’s competition ‘Humpback Whale Identification’
This time, I only spent one month on competition “Humpback Whale Identification”. But still, get a little step forward than previous competitions. Here are my summaries: 1. Do review ‘kernels’ in...
View ArticleUsing XGBoost to predict large sparse data
For using XGBoost to predict, I wrote code like this:test = load_npz('test.npz') test = csr_matrix(test, dtype = 'float32') xgb_model = xgb.Booster({'n_jobs': -1}) xgb_model.load_model(MODEL_PATH)...
View ArticleClik here to view.
Experiencing TensorCore on RTX 2080 Ti
My colleague’s bare metal PC with three-fans-RTX-2080-Ti My previous colleague Jian Mei has bought a new GPU – RTX 2080 Ti for training bird images of http://dongniao.net/. After he installed all the...
View ArticleProblems about using DistCp on Hadoop
After installing all Hadoop environment, I used DistCp to copy large files in distributed cluster. But it report error:#hadoop distcp Error: A JNI error has occurred, please check your installation and...
View ArticleSummaries for Kaggle’s competition ‘Histopathologic Cancer Detection’
Firstly, I want to thank for Alex Donchuk‘s advice in discussion of competition ‘Histopathologic Cancer Detection‘. His advice really helped me a lot. 1. Alex used the ‘SEE-ResNeXt50’. Instead, I used...
View ArticleClik here to view.
How to writing papers with Markdown
Last weekend I exported my Jupyter Notebook records into a PDF format file. Surprisingly, the PDF file looks so good that I begin to think about using Jupyter Notebook or Markdown instead of LaTex to...
View ArticleSome tips about PyTorch and Python
1. ‘()’ may mean tuple or nothing. len(("birds")) # the inner '()' means nothing len(("birds",)) # the inner '()' means tulple because of the comma The result is:5 1 2. Unlike TensorFlow’s static...
View ArticleClik here to view.
Using Single Shot Detection to detect birds (Episode one)
SSD (Single Shot Detection) is a type of one-stage object detection neural network which uses multi-scale feature maps for detecting. I forked the code from ssd.pytorch, and added some small...
View ArticleClik here to view.
Debugging the problem of ‘nan’ value in training
Previously, I was using CUB-200 dataset to train my object detection model. But after I used CUB-200-2011 dataset instead, the training loss became ‘nan’.iter 10 || Loss: 17.9996 || timer: 0.2171 sec....
View ArticleClik here to view.
Using Single Shot Detection to detect birds (Episode two)
In the previous article, I reached mAP 0.739 for VOC2007. After about two weeks, I add more tricks to reach mAP 0.740. The most important trick is escalating the expand-scale of augmentation which is...
View ArticleUse docker as normal user
I have used docker for more than 4 years, although not in product environment. Until last week, my colleague told that docker can be used as non-root user. The document is here. I just need tosudo...
View ArticleGet the type of engine for a table in MySQL
To view show the type of engine a MySQL table used, we could type:show table status where name='test';...
View ArticleClik here to view.
Some tips for opencv-python
Type conversion Using opencv-python to add object-detection rectangle for the image:import cv2 import matplotlib.pyplot as plt import numpy as np img = cv2.imread("./birds/Martens's...
View ArticleClik here to view.
Using Single Shot Detection to detect birds (Episode three)
In the previous article, I reached mAP 0.740 for VOC2007 test. After one month, I found out that the key to boost the performance of object detction is not only based on cutting edge model, but also...
View ArticleTips about Numpy and PyTorch
1. Type convertion in Numpy Here is my code:import numpy as np a = np.asarray([1, 2]) b = [] c = np.concatenate((a, b)) print(c.dtype)Guess what? The type of variable ‘c’ is ‘float64’! Seems Numpy...
View ArticleClik here to view.
Investigating about Streaming ETL solutions
Normal ETL solutions need to deliver all data from transactional databases to data warehouse. For instance, DBAs or Data Scientists usually deploy a script to export whole table from database to data...
View ArticleThe weird comparison behaviours of Python string
A part of my code didn’t work well, as below:if mapping and mapping['colour'] is 'Red': print('Special colour')It will not print out anything totally. So I directly printed out the actually value of...
View ArticleClik here to view.
Some ideas about building streaming ETL on AWS
After discussed with technical support guys from AWS, I get more information about how to use all the service of AWS to build a streaming ETL architecture, step by step. The main architecture could be...
View Article