Resources

Lab GitHub repos

We make our code publicly available. The following GitHub website contains our repositories, including solutions for data science competitions.

https://github.com/CIBR-Okubo-Lab

Skill sets

Here is a list of skillsets that are useful for our research.

	Essential	Optional
Computer vision	OpenCV, ffmpeg
Containerization	Docker
Data visualization	Matplotlib	Plotly, Seaborn
Deep learning	PyTorch, TensorFlow	JAX
Deep learning helper tools	PyTorch Lightning, Hydra	Hugging Face Transformers
Environment management	pip, conda
Experiment tracking	W&B
Hyperparameter tuning	W&B Sweeps	Optuna
Machine learning	Scikit-learn
Network analysis	NetworkX, graph-tool	igraph
Numerical computing	Numpy, Scipy, Matlab	Julia
OS	Linux
Programming	Python	C++, Java
Tabular data	Pandas	Polars
Version control	Git, GitHub

Textbooks

Here are some recommended textbooks and online resources for anyone who is interested in our type of work, which includes neural data analysis, machine learning, and computational neuroscience.

Data structures and algorithms

• Deep understanding of basic algorithms is essential to our work, and reading through a textbook is recommended to give you an overview. While some textbooks are more like encyclopedic collections of algorithms, this book stands out for its coherent narrative flow and explanations that explicitly spells out the thinking behind main ideas. Chinese version is also available.

• It is also useful to have algorithm textbooks that are focused on implementations. While there are plenty of online sources, it is still useful to have a formal reference that has consistent design and well-commented example code.

Machine learning and deep learning

• Machine learning in general

Machine Learning with PyTorch and Scikit-Learn.jpg

• “Machine Learning with PyTorch and Scikit-Learn” by Raschka: Sebastian Raschka publishes lots of great articles and books on deep learning and machine learning that contain clear explanations and diagrams. This book covers the basics of both classical machine learning (using Scikit-Learn) and deep learning (using PyTorch).

• “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow” by Géron: This book also covers the basics of classical machine learning (using Scikit-Learn) and deep learning (using Keras and TensorFlow). Each concept is explained clearly and it is a bit more comprehensive than the book above.

• “Mathematics for Machine Learning” by Deisenroth, Faisal, and Ong. This is an excellent book for reviewing parts of linear algebra, vector calculus, and probability that are relevant for machine learning. There are many clear diagrams and detailed step-by-step derivations. One thing to be aware of is that this book uses a numerator layout of the derivative, so the equations in the vector calculus sections might look different from what you might be used to.

• Probabilistic machine learning

• “Pattern Recognition and Machine Learning (PRML)” by Bishop: Even though this book was published before the deep learning era (2006), the concepts that are covered remain extremely important till this day. This book gives a solid foundation for probabilistic modeling and inference. In our lab, we deepened our understanding by trying to replicate all the figures in Ch 1-3, 9 just using Numpy and Matplotlib.

• Deep Learning

• “Dive into Deep Learning” by Zhang et al., 动手学深度学习: A great reference on deep learning techniques with helpful code snippets that are available in several different deep learning frameworks. Suitable for learning both concepts and how they are implemented.

• “Deep Learning” by Bishop & Bishop: Covers important concepts in deep learning by focusing on a selected number of important topics and explaining them in detail, similar in style to PRML by the same author (but less mathematical).

• “Understanding Deep Learning” by Prince: Covers the similar sets of topics as the Deep Learning book by Bishop & Bishop. It has many original diagrams that are helpful.

Computational neuroscience

• “Theoretical Neuroscience” by Dayan & Abbott: A classic textbook on computational neuroscience that covers multiple levels of brain modeling, from individual neurons to neural networks, and learning algorithms.

• Neuromatch computational neuroscience course: An excellent course covering various computational neuroscience topics with Python-based exercises. Originally designed as a 3-week online course, but the materials can also be studied at your own pace.