Basic Statistics Know-how

• Probability: definition and properties

• Conditional probability (all things “Bayes” are based on this concept, Bayes theorem, Bayes classifier, Naïve Bayes classifier, etc.)

• Methods of enumeration (this comes in handy for example in resampling techniques)

• Random Variables and random experiments (definition) (you need those when you are studying distributions)

• The concept of mathematical expectation (metrics in machine learning are expectations over the inputs, or in other words over the training data distribution so to speak)

• Discrete and continuous random variables

• Discrete distributions: Poisson and binomial

• Continuous distributions: normal, chi-square and gamma

Reading material

Hogg, Robert V., Elliot A. Tanis, and Dale L. Zimmerman. Probability and statistical inference. Upper Saddle River, NJ, USA:: Pearson/Prentice Hall, 2010.

Complete Chapter 1, Chapter 2: 2.1, 2.2, 2.4 and 2.6, Chapter 3: 3.1, 3.2 and 3.3

PDF Download: https://faculty.ksu.edu.sa/sites/default/files/677_fr37hij.pdf

If you are interested in a much more formal discussion of the topics, the following book is fantastic.

Casella, George, and Roger L. Berger. Statistical inference. Cengage Learning, 2021.

Chapter 1: 1.1, 1.2, 1.4, 1.5, 1.6, Chapter 3: 3.1, 3.2, 3.3

PDF Download: http://home.ustc.edu.cn/~zt001062/MaStmaterials/George%20Casella&Roger%20L.Berger--Statistical%20Inference.pdf

U. Michelucci, Statistics for Scientists - A Primer for Researchers and Professionals, Springer Briefs Collection, Springer Nature, 2024, (Available in June 2024)

Basic Algorithm Know-how

• The art of analysing algorithms and asymptotic notation

• Sorting algorithms (Insertion sort, Heapsort, Quicksort, etc.)

• Basic data structures: stacks and queues, linked lists, hash tables, binary search trees

Reading material

The bible for algorithms is

Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2009). Introduction to algorithms. MIT press.

Chapter 1 complete, Chapter 2 complete, Chapter 3 complete, Chapter 6: 6.1-6.4, Chapter 7: 7.1, 7.2, Chapter 10: 10.1, 10.2, Chapter 11: 11.1, 11.2, 11.3, Chapter 12: 12.1, 12.3

Basic Calculus and algebra

Calculus is important in machine learning, and at least a basic understanding is important. Optimisers as gradient descent use gradients, and in general the back-propagation algorithm is heavily based on derivatives of operations. So you should at least understand what a derivative is. The following concepts should be understood and mastered by any machine learning engineer.

• The concept of a function

• The main idea of optimisation (minimising a function generally and very imprecisely speaking)

• Derivatives (and partial derivatives) (as mentioned this becomes crucial for any optimizer and almost for any algorithm that use back-propagation)

• Integrals (they are very relevant in many statistical analysis, for example in evaluating expectations of continuous random variables)

• The concept of limit. It comes very useful as soon as you start looking at slightly more advanced concepts in statistics and machine learning.

• The basic concept of algebra, in particular matrices

• matrix multiplication

• inverting matrices

Reading material

Apostol, T. M. "Calculus. one-variable calculus with an introduction to linear algebra. vol 1.–john wilay and sons." Inc(1966).

Michelucci, U. “Fundamental Mathematical Concepts for Machine Learning in Science”, Springer Nature 2024, Text Book in Computer Science (Available in May 2024)

Programming languages and tools

Everyone, and I mean everyone, should master Python (and no, in R you cannot really develop neural networks believe me…). In particular is important to really understand well how Python works, its datatypes and also how to use object-oriented approaches (classes, etc.). Classes are often used when developing, for example, neural networks with TensorFlow or PyTorch. The following libraries should be mastered at an intermediate/advanced level:

• numpy

• matplotlib

• pandas

• TensorFlow (more on that on the page dedicated to neural networks)

You should also master git. And I mean you should be able to work comfortably with GitHub if you want to collaborate with other researchers.

How to write and read scientific papers

You should have a basic understanding of how to read and write a scientific papers, how they are structured, how to skim a paper and so on. I wrote this article to help beginners in the field of machine learning research (in other fields papers are structured differently). You may think that scientific papers are not relevant, since you are a developer, an engineer right? Wrong. Research results are communicated through papers (peer-reviewed and non). So you should know how to read and write them. No way around it.

I wrote this paper in 2021 to highlight the most important aspects of writing and reading a paper.
DOWNLOAD THE PAPER.