Research

From UNIDAPT.org

It may not always be visible to the IT users, but developing and optimizing new computing systems using current technology is too time consuming and costly. Our long term goal is to revisit over-complicated, under-performing and power-hungry computing technology, and develop self-tuning, adaptive computing systems based on gradual parametrization of all computing systems using collaborative, open-source design, characterization and optimization tools, on empirical automatic code/architecture optimization/characterization, and on continuous collection of multiple benchmarks, data sets and optimization/characterization information in open repositories. We actively research on how to systematize all available knowledge using various novel interdisciplinary statistical, collective and machine learning techniques within my research group and collaborative tuning center. We are always very goal-oriented to deliver quickly what we promise given enough R&D freedom.

Automating code, compiler and architecture design and optimization should help to improve performance, power consumption and reliability of computing systems, reduce their development cost and time to market by predicting better optimizations, architecture designs on run-time adaptation. We hope that this in turn will boost innovation and research in bio-informatics, medicine, physics, aeronautics, robotics, chemistry, finances, image/video processing and other important areas.

From time to time, we give invited talks about our techniques at companies and universities including Intel, IBM, UIUC, ICT, etc. Most of our techniques and tools have been published and references in conferences/journals such as ACM TACO, MICRO, PLDI, ASPLOS, CGO, PACT; referenced in various patents; included in the HiPEAC 2012-2020 research roadmap and are now actively used and extended in academia and industry (Intel, IBM, Google and others).

We are very grateful to Prof. Michael O'Boyle, Prof. Olivier Temam and Prof. Francois Bodin for a very special and long-term collaboration! We are also grateful to all our great colleagues for interesting and sometimes tough research discussions, feedback and collaboration.

• Main achievements:
  • Moved empirical automatic performance tuning, iterative optimization (compilation), statistical and machine learning techniques to real production compilers and computing systems, and started the whole system parametrization and abstraction to enable future self-tuning, adaptive computing systems. Designed Collective Tuning Center (cTuning.org) and led the development of the first publicly available machine learning enabled self-tuning compiler based on automatic correlation between program features, run-time behavior and optimizations (MILEPOST GCC (with MILEPOST colleagues) and cTuning CC) focusing at the moment on improving program execution time, code size, compilation time, power consumption and enabling run-time adaptation. Though this system is still in a preliminary state, we showed for the first time how to substitute the whole human-tuned state-of-the-art optimization heuristic of a production compiler with the automatically learned one across multiple programs and architectures using empirical iterative and collective optimization, program features and predictive modeling. cTuning/MILEPOST technology revolutionizes code and architecture design and optimization, helps to reduce power consumption and time to market for new computing systems, and is now used by major companies and in large US, EU and other international R&D projects. All developed tools have been released as open-source to the community to continue collaborative R&D to automate and simplify code and architecture design and optimization, enable sharing of optimization cases and improve the quality of academic research. cTuning/MILEPOST technology opens up many research opportunities, so it's an on-going, evolving project with many R&D topics available. Current collaborative projects include automating fine-grain/polyhedral optimizations, parallelization and run-time optimization and adaptation using statistical and machine learning techniques.

• • Developed UNIDAPT concept to statically enable run-time optimizations and self-tuning binaries through function cloning and integrated low-overhead program/system behavior monitoring routines. Such adaptive binaries can automatically learn from the past experience and adapt/react to various environments, run-time behavior and contentions to improve performance, power consumption, efficiency and cost of both embedded systems and HPC data centers (cloud computing). This framework opened up many research possibilities, has been used in multiple research projects in collaboration with UPC, ICT, IBM, CAPS Enterprise, STMicro, has been supported by MILEPOST, HiPEAC and Google Summer of Code grants, has been referenced in patents and has been extended to speed up iterative compilation (FCOP2005, FCOP2006), enable transparent continuous collective optimization (FT2010,FT2009,FMPP2007), enable portable program characterization techniques based on reactions to optimizations (FT2010,FT2009), enable predictive scheduling for heterogeneous multicore systems (JGVP2009), enable adaptive libraries based on dataset characterization using machine learning and decision trees (LCWP2009) among many other usages. Since 2007 it is being actively extended by Google Inc. for data centers (cloud computing).
  • Developed a novel and simple approach to characterize program memory/CPU behavior and quickly detect realistic lower bound of the execution time of memory intensive applications by modifying individual assembler instructions including converting array accesses to scalars in various ways without preserving the semantics of the code while avoiding code crashing to be able to directly compare original and transformed programs. This practical technique helps end-users and developers to focus their effort only on those parts of the code that can benefit from memory optimizations and opens up various other R&D opportunities (FOTP2004,FUR2004,FOTP2001,EOS tools).
  • Participated in the preparation of the HiPEAC research vision (2012-2020) (reviewing and writing parts on automatic self-adaptation, split-compilation and collective optimization).
  • Initiated the International Workshop Series on Statistical and Machine learning approaches to ARchitecture and compilaTion (SMART) together with my colleagues from the University of Edinburgh.

• Main research interests and expertise:
  • Statistical analysis, machine learning, data mining, multi-objective optimizations - we believe that this techniques will help to simplify, systematize and automate code and architecture design, characterization and optimization, so we use combination of those techniques in most of our projects.
  • Collaborative research infrastructure and repositories - initiated development of open source R&D tools and repositories with unified interfaces for gradual parametrization of all computing systems to automate, simplify and systematize code and architecture design, characterization and multi-objective optimization (Related publications: FT2010, FKMP2011, FT2009, Fur2009, HPWP2010, FMTP2008, FCOP2007; Related tools: cTuning tools, cTuning repository).
    
    Collective Tuning Center (cTuning.org) is a community-driven open-source collaborative tools and repositories to simplify, systematize and automate code and architecture design, characterization and optimization. It enables sharing of benchmarks, datasets, optimization cases through unified web-services and tools with common APIs to be able to predict better optimizations or architecture designs provided there is enough information collected in the repository from multiple users. cTuning should enable the future smart self-tuning adaptive systems (including programs, compilers, run-time systems, architectures) based on empirical compilation and performance evaluation, statistical collective optimization and machine learning as well as considerably reducing time to market and improving the quality and reproducibility of the research.
  • Interactive, plugin-based compilers - designed unified API and light-weighted plugin system to transform production-quality compilers with the state-of-the-art transformations into powerful interactive collaborative research research infrastructure with minimal changes to avoid building it from scratch and reuse functions for program analysis, fine-grain optimizations, instrumentation, etc or to help to clean up and systematize old, messy compilers (Related publications: FKMP2011, FT2010, MCFP2010, YYLP2010, HPWP2010, FT2009, Fur2009,FMTP2008, FCOP2007, FCOP2005; Related tools: cTuning CC, MILEPOST GCC, ICI).
  • Adaptive, machine-learning based self-tuning compilers - developed techniques to adapt compiler optimization heuristic for multiple-objectives (execution time, compilation time, code size, etc) for a given program and and architecture automatically based on empirical, probabilistic, iterative optimization, predictive modeling (machine learning), program static and dynamic features, collective optimization repositories, and off-line training (Related publications: FKMP2011, FT2010, MCFP2010, YYLP2010, FT2009, DJBP2009, LCWP2009, Fur2009, FTMP2008, LCFP2007, FMPP2007, DCFP2007, CFAP2007, FC2007, FCOP2007, FCOP2006, ABCP2006, FCOP2005, FOTP2005, LF2005, FUR2004, FOK2002; Related tools: cTuning CC, MILEPOST GCC, cTuning repository, cTuning optimization predictor).
  • Collective, multi-objective optimization - enabled transparent optimization knowledge reuse among different programs, data sets, architectures from multiple users to find "good" optimizations or tune compiler optimization heuristic using statistical analysis and collecting characterization and optimization data in an open repository (Related publications: FT2010, FKMP2011, YYLP2010, FT2009, LCWP2009, FMTP2009, Fur2009, FMPP2007, FC2007, FCOP2007, FCOP2006, FCOP2005, FOTP2005, FUR2004, FOK2002; Related tools: cTuning tools, cTuning repository).
  • Adaptive, self-tuning programs - enabling run-time adaptation of static and dynamic programs based on static multi-versioning and statistical and machine learning techniques (Related publications: FT2010, FKMP2011, FT2009, LCWP2009, Fur2009, FMPP2007, FCOP2006, FCOP2005; Related tools: cTuning tools).
  • Adaptive, self-tuning libraries - enabled self-tuning libraries for uni-core and heterogeneous multi-core systems based on static multiversioning and predictive modeling (Related publications: JGVP2009, LCWP2009, Fur2009, FMPP2007, FCOP2006, FCOP2005; Related tools: cTuning tools).
  • Memory characterization through assembler modifications - developed novel technique for fast and accurate memory characterization using various assembler-level code modifications without preserving the semantics of the code to detect where to focus optimizations (Related publications: DJBP2009,DCFP2007,FOTP2004,FUR2004,FOTP2001; Related tools: Edinburgh Optimizing Software - outdated and moved to cTuning.org).
  • Adaptive parallelization for heterogeneous computing systems - enabled auto-parallelization and run-time scheduling on heterogeneous multi-core systems based on statistical and machine learning techniques (Related publications: JGVP2009, Fur2009, LFF2007).
  • Collective Benchmarks - designed/collected multiple benchmarks and kernels with multiple datasets to enable realistic performance evaluation and program/dataset/architecture characterization (Related publications: YYLP2010, Fur2009, FCOP2007; Related tools: cBench/MiBench/MiDataSets).
  • Automated architecture design space exploration - investigated techniques for continuous exploration of different architectural designs (statically or at run-time using reconfiguration) to find the best designs for performance, power consumption, fault-tolerance (Related publications: DJBP2009, CDAP2006, DFGP2007).
  • Split-compilation - researched various techniques to find balance between static and dynamic optimizations using predictive modeling combined with off-line and on-line training and annotations (Related publications: LCFP2007).
  • Polyhedral optimizations - initiated development of practical, adaptive techniques to combine polyhedral program representation and compiler optimization/parallelization to overcome the limitations of current compiler transformations (Related publications: LF2009, LF2005).

• Software development:
  • We are developing open-source research tools with common APIs to support our research and enable collaborative and replicable R&D. Our activities are supported by academia and industry including multiple HiPEAC partners. If you are interested to help us develop common research tools with unified APIs (compilers, architecture simulators, run-time environments, etc) you are warmly welcome to join our cTuning effort.
  • More details about our collaborate software developments can be found at our Developments page.

• Dissemination:
  • All publications.

• Collaboration:
  • Université de Versailles-Saint-Quentin-en-Yvelines (France), UIUC (USA), ICT (China), University of Edinburgh (UK), UPC (Spain)
  • Intel, CAPS Enterprise, GENCI, CEA, STMicroelectronics, IBM

• Software development:
  • We are developing open-source research tools with common APIs to support our research and enable collaborative and replicable R&D. Our activities are supported by academia and industry including multiple HiPEAC partners. If you are interested to help us develop common research tools with unified APIs (compilers, architecture simulators, run-time environments, etc) you are warmly welcome to join our cTuning effort.
  • More details about our collaborate software developments can be found at our Developments page.

If you would like to join our group, this page lists current positions and requirements.
If you would like to contact or visit us you can find more details here.