Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Architecture

• Biological versus artificial neurons
• The structural model of an ANN
• Activation functions employed in ANNs
• Common classes of network architectures

Mathematical Foundations and Learning Mechanisms

• Revision of vector and matrix algebra
• State-space concepts
• Principles of optimization
• Error-correction learning
• Memory-based learning
• Hebbian learning
• Competitive learning

Single Layer Perceptrons

• Structure and training of perceptrons
• Introduction to pattern classifiers and Bayes' classifiers
• Utilising perceptrons as pattern classifiers
• Perceptron convergence
• Constraints of perceptrons

Feedforward ANNs

• Structure of Multi-layer feedforward networks
• The Back propagation algorithm
• Back propagation: training and convergence
• Functional approximation via back propagation
• Practical and design considerations for back propagation learning

Radial Basis Function Networks

• Pattern separability and interpolation
• Regularization Theory
• Regularization and RBF networks
• RBF network design and training
• Approximation characteristics of RBF

Competitive Learning and Self-Organizing ANNs

• General clustering procedures
• Learning Vector Quantization (LVQ)
• Competitive learning algorithms and architectures
• Self-organising feature maps
• Characteristics of feature maps

Fuzzy Neural Networks

• Neuro-fuzzy systems
• Background in fuzzy sets and logic
• Design of fuzzy systems
• Design of fuzzy ANNs

Applications

• A discussion of selected Neural Network applications, highlighting their advantages and associated challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent case
  • Guarantees for finite hypothesis sets – inconsistent case
  • Generalities
    • Deterministic versus stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection
• Raudeeisher Complexity and VC Dimension
• Bias-Variance tradeoff
• Regularisation
• Over-fitting
• Validation
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self-Organisation Maps (SOM)
• Kernel-induced vector space
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This section builds upon the concepts covered in Days 1 and 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demos and Applications