!R-Wonc'19
というのを聴講．
https://usability-research.r-ccs.riken.jp/r-wonc19/

::Giacomo Indiveri, Neural processing and learning electronic circuits for building neuromorphic cognitive agents
* intro
** driven by AI
** driven by Big-Data https://www.nature.com/articles/d41586-018-01683-1
* Bee brain specs
** 1mg, 1mm^3, 960,000 neurons, 10^{-15} J/spike energy/op
* Neuromorphic processor chips
** spikes in, and spikes out
** analog subthreshold circuits
** inhomogeneous, imprecise, and noisy,
** massively parallel
* DYNAP-SEL-Dynamic Neuromorphic Asynch Processor with Self Learning
* Adaptive CardioRespiratory Pacemaker EU project

::Mike Davies, A New Era of Neuromorphic Computing
* cf. https://newsroom.intel.com/editorials/intel-creates-neuromorphic-research-community/
* Background (pict.)
* The Engineering Perspective (pict.)
* LOIHI: https://www.computer.org/csdl/magazine/mi/2018/01/mmi2018010082/13rRUIJcWtw
** https://www.researchgate.net/publication/322548911_Loihi_A_Neuromorphic_Manycore_Processor_with_On-Chip_Learning
** 128 neuromorphic cores, 128k neurons, 128M synapses
** 14nm FinFET
** cf. http://niceworkshop.org/wp-content/uploads/2018/05/Mike-Davies-NICE-Loihi-Intro-Talk-2018.pdf
* Loihi systems
** wolf mountain, Nahuku, Kapoho Bay, ...
*** cf. https://converge360.com/Blogs/Future-Tech-Blog/2018/12/NeuroBiological-USB-Intel.aspx
*** cf. https://www.top500.org/news/intel-ramps-up-neuromorphic-computing-effort-with-new-research-partners/
* SNN algorithms discovery and development
* Speech Recognition: Keyword spotting
** https://arxiv.org/abs/1812.01739
* Spiking LCA dynamics
* Spike-based LSTMs  LSNN
** cf. Long short-term memory and learning-to-learn in networks of spiking neurons https://arxiv.org/abs/1803.09574
** cf. Biologically inspired alternatives to back-propagation through time for learning in recurrent neural nets https://arxiv.org/abs/1901.09049
** cf. Adaptive Control of a Robot Arm using Loihi https://royalsocietypublishing.org/doi/full/10.1098/rspb.2016.2134
* Graph search
* Olfacton-Inspired One Shot Learning
** cf. olfaction inspired machine learning https://arxiv.org/pdf/1802.05405.pdf
* Why Spikes: (pic.)

:: Cerebellum
* a little brain
** cf. Cerebellar ataxia - https://www.youtube.com/watch?v=Txlvuu2byUY
* simulation by CPU, GPU, FPGA, PEZY
** GPU, FPGA, PEZY - realtime simulation
* human-scale cerebellum on K
** 68 billion neurons on 82,944 nodes
** MONET (in-house simulator)
** 600 time slower than realtime (10min. for 1s)
* application
** arm control
** reinforcement learning in cerebellum
*** reinforcement learning can go in parallel (massively)
***- cf. Hybrid Reward Architecture for Reinforcement Learning https://arxiv.org/abs/1706.04208
***- cf. Hybrid Reward Architecture https://github.com/Maluuba/hra

:: A Benchmarking and Programming Framework for Spiking Neuromorphic Computing Systems
A Survey of Neuromorphic Computing and Neural Networks in Hardware - https://arxiv.org/abs/1705.06963

* 3 examples (pict.)
* TENNLab sotware framework
** https://www.semanticscholar.org/paper/The-TENNLab-Exploratory-Neuromorphic-Computing-Plank-Schuman/95a139d59d7551128e89b62b9d114ff1c8a27c09
** http://neuromorphic.eecs.utk.edu/publications/2018-08-17-the-tennlab-exploratory-neuromorphic-computing-framework-submission/
* Types of Neuromorphic Implementations
** DANNA 2- fuly digital https://dl.acm.org/citation.cfm?id=3229894
** mr DANNA - mixed analog-digital
** SOEN - optoelectronic

::VLSI Research for Neuromorphic Computing in IBM Research
* spiking/non-spiking neural network, digital/analog implementation
** spiking/digital - TrueNorth
** non-spiking/digital - GPGPU, FPGA, FPU arrays
** spiking/analog - Spiking neural network chips w/ non-volatile memory arrays
** non-spiking/analog - 
* A Scalable Multi-TeraOPS Deep Learning Processor Core for AI Training and Inference  - https://ieeexplore.ieee.org/document/8502276, https://xpressdrivein.org/glo16/pdf/C04-2.PDF
** approximate computing by reduced precision computations
* A million spiking-neuron integrated circuit with a scalable communication network and interface - http://science.sciencemag.org/content/345/6197/668 http://paulmerolla.com/merolla_main_som.pdf
** 1M neurons, 256M synapses tileable 2D-onchip
* NVM synaptic array - eg. phase change memory (pict.)
* Analog multiply accumulation with non-volatile memory array
* NVM Weight Variation Imact on Analog Spiking Neural Network Chip - https://link.springer.com/chapter/10.1007/978-3-030-04239-4_61
* Lightweight Refresh Method for PCM-based Neuromorphic Circuits - https://www.semanticscholar.org/paper/Lightweight-Refresh-Method-for-PCM-based-Circuits-Ito-Ishii/5ae6ddfcf1820a7c1289d97197c09b598bc0724a

:: AI on the Edge: Frontiers for Energy-Efficient Hardware Architectures
* "Structure" is a key
* Binary/Ternary DNN accelerator VLSI 2017
** Binary/Ternary, Reconfigurable in Memory
** cf. https://www.researchgate.net/publication/321930684_BRein_Memory_A_Single-Chip_BinaryTernary_Reconfigurable_in-Memory_Deep_Neural_Network_Accelerator_Achieving_14_TOPS_at_06_W
* Log-Quantized DNN accelerator with 3D SRAMs
** QUEST(Log QUantization, MIMD Parallel Engine, Die-STacking with SRAMs)
** cf. Convolutional Neural Networks using Logarithmic Data Representation - https://arxiv.org/pdf/1603.01025.pdf
* Dynamically reconfigurable processor with AI-MAC engine
** DRP(96-core, 333MHz) + AI-MAC(1024, 500MHz)
* The Era of "Intelligence at the Edge" will Begin
** Common Key Features: Mostly static, dataflow rich, (self) evolvable
** Procedure Oriented Computing -> Structure Oriented Computing: Reconfigurable HW, "Virtualized" Reconfigurable HW (post FPGA), Dataflow Oriented Machine (w/ reduced synth. cost)

:: Towards biologically plausible learning of spike-based cognition
* CT-AuGMEnt

::Stochastic Computing for Brainware LSI


:: 関連する？
* Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain - https://www.frontiersin.org/articles/10.3389/fnins.2018.00891/full
* Adaptive motor control and learning in a spiking neural network realised on a mixed-signal neuromorphic processor - https://arxiv.org/pdf/1810.10801.pdf
* Dual Supervised Learning - https://arxiv.org/abs/1707.00415
* Implementation of a Liquid State Machine with Temporal Dynamics on a Novel Spiking Neuromorphic Architecture - https://www.osti.gov/servlets/purl/1405258