Data Science - ML - 2019 Round-up
Nulldata Edition #21
Hey Reader, As we’re winding up the year, I thought of sharing a few things that’s available on Top of my mind that popped up as something special in this Calendar year 2019.
NLP NLP (also NLG)
Natural Language Processing - took a huge leap in 2019. Whether you are an NLP Practioner or Not, You can’t ignore the fact how much models in NLP has progressed. Especially how things peaked up since the arrival of BERT (not the one from Sesame Street) - transformer-based model. Things got craizer with companies like `spacy` and `hugging face` 🤗 doing great open-source contribution. Then there was OpenAI’s GPT-2 that gained huge media attention for all wrong reasons.
BERT - https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html
BERT Visual Guide - https://jalammar.github.io/a-visual-guide-to-using-bert-for-the-first-time/
🤗 - Transformers - https://github.com/huggingface/transformers
spacy v2.2 - https://explosion.ai/blog/spacy-v2-2
Open AI Language Models - https://openai.com/blog/better-language-models/#update
GANs
If there was one profession that I thought would never be replaced by AI, It was actors/models but that thought was smashed to the deepest part of the earth with the rise of GANs. GANs were also useful in gaining Media interest.
Style GAN - https://github.com/NVlabs/stylegan
Cycle GAN - https://github.com/junyanz/CycleGAN
GANs intro. by Ian Goodfellow
OpenAI Generative Models - https://openai.com/blog/generative-models/#gan
Deep Learning in General
`xgboost` was once the King of Kaggle Leaderboard but right now it’s all about Deep Learning. But Deep Learning has even got out of Kaggle and entered mainstream with things like detecting diseases. Deep Learning Frameworks such as Tensorflow and Pytorch got huge updates and also fans (also flame wars). Google also attempted to take Tensorflow out of Python ecosystem with Swift for TF and TF.js. And, It’d be a crime if this para doesn’t include Jeremy Howard and Rachel Thomas’ Fast.ai efforts in AI democratization and Inclusion.
Making Neural nets uncool again - https://www.fast.ai/
TF 2.0 - https://blog.tensorflow.org/2019/01/whats-coming-in-tensorflow-2-0.html
Why Swift for TF - https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md
Pytorch new Tools - https://pytorch.org/blog/pytorch-adds-new-tools-and-libraries-welcomes-preferred-networks-to-its-community/
TF.js - https://www.tensorflow.org/js
Rise of Auto ML Tools
2019 has been a great year for startups with AI-focus and also Meta-AI. Companies like H2O.ai and Datarobot focusing on AutoML raised good funds from Venture Capitalists. Deep Learning Wrappers like Keras and Fastai have been the Go-To strategy for many Deep Learning Practioners. Google also has got quite a few tricks in this area in their Google Cloud Platform with AutoML Vision. Recently AWS joined the party with AWS Sagemaker AutoPilot and autogluon (open-source python library)
https://www.datarobot.com/news/press/datarobot-announces-206-million-series-e-funding-round/
Autogluon MXnet - https://autogluon.mxnet.io/
Importance of IML / Explainable AI
As much as AI tools and innovations gained positive media attention, they also gathered some criticism and fear (which most of the times were simply media hyped). But that laid strong foundation to an important section of ML and AI which is Interpretable Machine Learning or Explainable AI. It’s all about emphasing that the models built should be understood by Humans. This gave rise to several IML tools, Books, Videos, Talks and Discussions. Machine Learning Fairness, Accountability and Bias became an actual thing sometimes taking forms as Code rather simply on Paper.
Awards
* AI Godfathers awarded Turing Award of the year ( a testament to their work and also how much important their work is becoming in CS)
* Hadley Wickham received COPSS Award
Thank you for being a subscriber, sharing feedbacks and motivating me to write this stuff!
Regards,
Abdul Majed Raja