Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Performance Concepts and Metrics
- Latency, throughput, power consumption, and resource utilisation
- Distinguishing between system-level and model-level bottlenecks
- Profiling strategies for inference versus training
Profiling on Huawei Ascend
- Leveraging CANN Profiler and MindInsight
- Diagnosing kernel and operator performance
- Managing offload patterns and memory mapping
Profiling on Biren GPU
- Utilising Biren SDK performance monitoring features
- Exploring kernel fusion, memory alignment, and execution queues
- Implementing power and temperature-aware profiling
Profiling on Cambricon MLU
- Using BANGPy and Neuware performance tools
- Gaining kernel-level visibility and interpreting logs
- Integrating MLU profilers with deployment frameworks
Graph and Model-Level Optimisation
- Strategies for graph pruning and quantisation
- Operator fusion and restructuring computational graphs
- Standardising input sizes and tuning batch parameters
Memory and Kernel Optimisation
- Optimising memory layout and data reuse
- Managing buffers efficiently across different chipsets
- Applying platform-specific kernel-level tuning techniques
Cross-Platform Best Practices
- Achieving performance portability through abstraction strategies
- Developing shared tuning pipelines for multi-chip environments
- Case study: Optimising an object detection model across Ascend, Biren, and MLU
Summary and Next Steps
Requirements
- Hands-on experience with AI model training or deployment pipelines
- A solid grasp of GPU/MLU compute principles and model optimisation strategies
- Familiarity with performance profiling tools and key metrics
Target Audience
- Performance engineers
- Machine learning infrastructure teams
- AI system architects
