Ali Beikmohammadi

Research

  • Reinforcement Learning
  • Deep Learning
  • Distributed / Federated Learning
  • Cyber-Physical Systems
  • Stochastic Optimization
  • Telecommunications
  • Computer Vision
  • Image Processing
Doctoral thesis

Toward Sample-Efficient Reinforcement Learning: Theoretical Foundations and Algorithms

PhD in Computer and Systems Sciences · Department of Computer and Systems Sciences (DSV), Stockholm University
Supervisor: Sindri Magnússon
Defence: 16 Oct 2026

Research themes

Sample-Efficient Reinforcement Learning

The core of my thesis: making RL converge faster by exploiting structure that is already available — teacher signals, prior knowledge, planner guidance, or inductive bias. Teacher-assisted exploration and pseudo-label-driven actor-critic acceleration cut convergence time substantially without changing the underlying algorithm class.

  • exploration
  • actor-critic
  • reward shaping
  • sample efficiency
  • delayed rewards

TA-Explore (AAMAS 2023) Accelerated Actor-Critic (Inf. Sciences 2024) Human-Inspired RL (J. Supercomputing 2025) NARS vs. Q-learning (AGI 2023) BLADE-TD (under review)Rule-Based Grid World (under review)

Federated & Distributed Optimization

Convergence guarantees where the convenient assumptions fail — without data similarity, under compressed and clipped gradients, under model mismatch between clients, and under adversarial participants. Theory paired with reference implementations, extending to distributed inference and resource-aware serving in edge–cloud settings.

  • federated learning
  • compression
  • biased gradients
  • convergence
  • decentralized learning
  • edge AI

FL without Data Similarity (IEEE TBD 2024) Distributed SGDM (IEEE TCNS 2025) CompFedRL (ECML-PKDD 2024) FedTD(0) (ECML-PKDD 2025) Trust-Aware Decentralized FL (preprint 2026) Collaborative Distributed Inference (preprint 2026) Generative Markov Model (preprint 2026)

Reinforcement Learning for Control & Cyber-Physical Systems

Learning-based control where safety, partial observability and physical constraints matter: multi-agent deep RL for smart power converters with Hitachi Energy and KTH, model-predictive-control-guided RL for power electronics, and adaptive traffic signal control under noisy, nonstationary conditions.

  • multi-agent RL
  • MPC
  • partial observability
  • smart converters
  • power systems

PriPG-RL (preprint 2026) Active Inference for Traffic Control (IEEE WF-IoT 2026) Energy-Efficient Cloud Data Centres (Trans. Machine Intelligence 2020) MPC-Guided Fast RL (under review)

Agentic & LLM-Based Systems

How to make systems built on large language models trustworthy enough to deploy: auditing fault containment in LLM-driven agents, grounding generation in retrieved evidence for high-stakes scientific prediction, sample-efficient test-time scaling, and the socio-technical questions agentic AI raises. This is also the research strand closest to my industry work.

  • LLM agents
  • RAG
  • evaluation
  • test-time scaling
  • reliability

Socio-Technical Agentic AI (preprint 2025) GICA (TMLR 2026)Fault Containment in LLM–RL Agents (under review)Evidence-Grounded AMP Prediction (under review)

Learning from Imbalanced & Incomplete Data

Systematic study of resampling methods — SMOTE, ADASYN and alternatives — against gradient-boosted and ensemble classifiers across imbalance regimes, alongside cost-sensitive Transformers for industrial prognostics and clustering that tolerates up to 50% missing data. Developed in part with Scania CV AB and Linköping University.

  • imbalanced learning
  • resampling
  • cost-sensitive learning
  • missing data
  • predictive maintenance
  • transformers

Cost-Sensitive Transformer (Cluster Computing 2026) RF & XGBoost under Imbalance (Technologies 2025) Churn Prediction Review (MLKE 2025) SMOTE & ADASYN in Telecom Churn (ICWR 2024)

Computer Vision & Multimodal Learning

Plant and leaf identification with deep CNNs and Vision Transformers, automatic multimodal fusion, and semantic segmentation for autonomous driving — several strands developed together with the Master's students I supervise, and several leading to joint publications.

  • CNN
  • Vision Transformers
  • multimodal fusion
  • segmentation

SWP-LeafNET (ESWA 2022) Multimodal Plant ID (Frontiers Plant Sci. 2025) Human Activity Recognition (ICSPIS 2019) Leaf Transfer Learning (ICSPIS 2018) Autonomous Driving Segmentation (CSICC 2021) Hierarchical Kannada-MNIST (CSICC 2021) ML for PEDOT Conductivity (Phys. Rev. Materials 2025)

Research projects

6

Code & reproducibility

10

Reference implementations for the papers. Updated automatically from GitHub.

Accelerated-Actor-Critic

—

Python★ 4updated 25 Jan 2024

TA-Explore

—

Python★ 1updated 15 Aug 2025

FedAlgo_WO_DataSim

—

Python★ 2updated 12 Jul 2024

DistributedSGDM

—

Python★ 1updated 30 Nov 2023

CompFedRL

—

Jupyter Notebookupdated 4 Jun 2024

FedRL

—

Pythonupdated 27 May 2025

OpenNARS-for-Applications

General reasoning component for applications based on NARS theory.

C★ 1updated 4 Sep 2023

Cost-Sensitive-Transformer-Model

Cost-Sensitive Transformer Model for Prognostics and Health Management

PureBasicupdated 11 Mar 2026

MultimodalPlantClassifier

Source code for the master's thesis by Alfreds Lapkovskis & Natalia Nefedova (2024): Advancements in Agriculture: Multimodal Deep Learning for Enhanced Plant Identification. Developing a Multi-organ Plant Classifier Using Multimodal Fusion Architecture Search

updated 19 May 2024

ML4TransferIntegral

—

Jupyter Notebookupdated 2 Aug 2025

Other projects

25
  • Intelligent licence plate recognition for the Municipality of Hamedan Industrial design and deployment of an automatic licence plate recognition system for municipal use, in cooperation with the city.
    • Computer Vision
    • Deployed System
    • OCR
  • Energy-aware cloud data centre scheduling Metaheuristic algorithm reducing data-centre energy use 27% at equal performance.
    • Optimization
    • Cloud
  • Deep RL for resource allocation in cognitive femtocell networks DQN-based allocation increasing network capacity by 34%.
    • DQN
    • Telecommunications
  • MaxViT leaf disease detection Vision Transformer reaching 98.25% accuracy on leaf disease classification.
    • Vision Transformers
    • Agriculture

    Master's thesis, as main supervisor

  • Speedy Double Q-learning Novel variant of double Q-learning improving convergence rates in value-based RL.
    • Reinforcement Learning
    • Q-learning

    Master's thesis, as main supervisor

  • Transformer-based seizure detection on EEG Transformer model for detecting epileptic seizures from EEG recordings.
    • Healthcare
    • EEG
    • Transformers

    Master's thesis, as main supervisor

  • Federated learning for decentralised crop analysis FedProx and FedAvg applied to distributed agricultural data without centralising it.
    • Federated Learning
    • Agriculture

    Master's thesis, as main supervisor

  • Fairness auditing in healthcare machine learning PCA/XGBoost method enforcing equalized odds difference and ratio in SVM and logistic regression models applied to healthcare decisions.
    • Fairness
    • Healthcare
    • Responsible AI

    Master's thesis I co-supervised

  • Deep RL for vehicle fuel economy SAC-based reinforcement learning framework reducing vehicle fuel consumption by 15%.
    • Reinforcement Learning
    • SAC
    • Automotive

    Master's thesis I co-supervised

  • Deep RL for active flow control PPO applied to active control of ultra-low Reynolds number flows, addressing complex fluid dynamics with learned policies.
    • Reinforcement Learning
    • Fluid Dynamics
    • PPO

    Master's thesis I co-supervised

  • Deep RL for drug discovery Reinforcement learning for candidate selection and efficacy prediction in drug discovery.
    • Reinforcement Learning
    • Drug Discovery
    • Healthcare

    Master's thesis I co-supervised

  • Personalised treatment recommendation for obstructive sleep apnoea Decision Tree, Random Forest and XGBoost ensemble raising treatment efficacy to 87%.
    • Healthcare
    • XGBoost

    Master's thesis I co-supervised

  • Hierarchical RL for network routing Boosted throughput 29% over baseline routing policies.
    • Reinforcement Learning
    • Networking

    Master's thesis I co-supervised

  • Air pollution forecasting with GRU and LSTM RNN architecture improving forecasting accuracy 8%, supporting timelier public-health response.
    • Time Series
    • RNN

    Master's thesis I co-supervised

  • TactileNet — surface roughness classification from EEG CNN classifying tactile surface roughness from EEG recordings.
    • EEG
    • CNN

    Team project

  • 3D MRI brain tumour segmentation using deep learning Volumetric segmentation of tumour regions in 3D MRI scans.
    • Medical Imaging
    • Segmentation

    Team project

  • Facial expression classification with transfer learning Transfer-learning CNN for classifying facial expressions.
    • Computer Vision
    • Transfer Learning

    Team project

  • Video human activity recognition with MobileNet Activity recognition in video using transfer learning from MobileNet.
    • Computer Vision
    • Transfer Learning

    Course project

  • Housing price modelling with a Time-aware Latent Hierarchical Model Time-aware latent hierarchical approach improving housing price prediction accuracy by 18%.
    • Time Series

    Course project

  • Speech enhancement using kernel decomposition Deep neural network with kernel decomposition for speech enhancement at reduced model cost.
    • Speech
    • Deep Learning

    Team project

  • Bimodal learning for multi-sensory effects synchronisation Bimodal deep learning model aligning sensory effects with audiovisual content.
    • Multimodal
    • Neural Networks

    Course project

  • Deep CNN denoiser priors for image restoration Learned CNN denoiser used as a prior within an image restoration pipeline.
    • Image Restoration
    • CNN

    Course project

  • Separable non-local means for image denoising Improvement to the separable non-local means algorithm for faster denoising.
    • Image Processing

    Course project

  • Recursive least squares for Wiener model identification Fast iterative RLS algorithm for identifying highly nonlinear Wiener models.
    • System Identification

    Team project

  • IoT greenhouse environment control Smart sensor network on ESP8285 for monitoring and controlling greenhouse conditions.
    • IoT
    • Embedded

    Team project

Honours and awards

11
  • Rhodins, Elisabeth and Herman, Memory Scholarship
  • Lars Hierta Memorial Foundation Scholarship
  • Rhodins, Elisabeth and Herman, Memory Scholarship
  • Outstanding Paper Award, 5th ICSPIS'19 conference
  • Selected as a talented student for direct PhD admission in Electrical Engineering (no entrance exam)
    Iran National Elites Foundation & Amirkabir University of Technology
  • Ranked 1st in cumulative GPA among all Electrical Engineering MSc students
    Amirkabir University of Technology — GPA 19.77 / 20
  • Selected as a talented student for direct MSc admission (no entrance exam)
    Tarbiat Modares University, Shahid Beheshti University, and Iran University of Science and Technology
  • Ranked 1st in cumulative GPA among all Electrical Engineering BSc students
    Bu-Ali Sina University — GPA 19.10 / 20
  • Member, Iran National Elites Foundation
  • Selected as an educational talented student, three consecutive years
    Bu-Ali Sina University
  • Board member, Scientific Association of Electricity
    Bu-Ali Sina University

Other collaborations

4
  • Universitat Pompeu Fabra (UPF), Barcelona Visiting PhD student, Artificial Intelligence and Machine Learning research group
  • SINTEF AS & KTH Mitigating toxicity and bias in large language models via reinforcement learning
  • KAUST Federated temporal difference learning under model mismatch
  • Karolinska Institutet Health informatics teaching and Master's supervision

Community service

  • Organizing Team Member — 2025 IEEE World Congress on SERVICES (SERVICES 2025)
  • Local Chair — Symposium on Intelligent Data Analysis (IDA 2024)
  • Organizing Team Member — 4th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS 2018)

Journal reviewing

  • IEEE/ACM Transactions on Networking
  • IEEE Communications Letters
  • Information Sciences
  • Expert Systems with Applications
  • Neural Networks
  • Pattern Analysis and Applications
  • Journal of Big Data
  • Artificial Intelligence Review
  • Mobile Networks and Applications
  • Frontiers in Plant Science
  • The Imaging Science Journal
  • The Journal of Supercomputing
  • Engineering Applications of Artificial Intelligence
  • BMC Medical Imaging
  • Cluster Computing
  • Complex Systems Informatics and Modeling Quarterly

Conference reviewing

  • NeurIPS
  • ICLR
  • ECML-PKDD
  • ECAI
  • IJCNN
  • ECC