Why JSON lost to Google Protocol Buffers? How MicroServices Communicate Efficiently Together?

In this video, we understand how is response data encoded during inter-network communications and also how is it stored on disk?
Sets, Maps, Arrays - they are in-memory data structures. But how can we store them on disks? Python uses in-built pickle module to encode (serialize) data to store them onto disks. But can it be read by a Java client?
Things would have been easy if we used JSON to encode data and store on disks. So why use Pickle?
Actually, we rely on other binary encodings like Google Protobuf, Facebook Thrift for doing this encoding and decoding.
We can serialize the data into protobuf in Python and then parse and read it from a C++ client - no issues!
Plus, we save on the amount of bytes needed to store the data. For a JSON taking 83 bytes on disk, protobuf may need only 33 bytes! Smaller sizes also result in faster inter-server communications. Thus, when a microservice talks to microservice, its common to encode response into protobuf which is then parsed by the reader.

The other benefit with Protobuf is it supports forward and backward compatibility - you can extend and add new keys to your data and old code can work with new data (forward) while new code can work with old data (backward).

One thing to note is Protobuf works well with messages which are few megabytes in size.

References
DDIA Book: https://amzn.to/3B6TJcm
System Design Playlist: https://bit.ly/system-design-course
Google Protobuf: https://developers.google.com/protocol-buffers/docs/overview
Blog1: https://auth0.com/blog/beating-json-performance-with-protobuf/
Blog2: https://betterprogramming.pub/use-binary-encoding-instead-of-json-dec745ec09b6


Sponsored By Educative.io
✅ 𝗘𝗱𝘂𝗰𝗮𝘁𝗶𝘃𝗲.𝗶𝗼 [46% OFF] 👉🏻https://bit.ly/educative-unlimited-extra-discount
Educative.io coupon "rachit" to get extra discount or use the link above to get "Unlimited" access to "ALL" courses on their websites - React, System Design, Coding Interview in C++, Python, Java, Redux, Docker and what not!

Chapters
00:00 Preview of Google Protobuffers Beating JSON
00:15 Sponsor
00:50 Introduction
1:12 How are Arrays, Sets, Maps Stored on Disk?
1:49 Encoding and Decoding in the Web HTTP Requests
2:36 Encoding Algorithm should support Decoding Algorithm
3:05 Approach A - Encode Everything to JSON!
3:38 Approach B - Binary Encodings Beat JSON for TerraBytes of Data
4:41 File Encoding Should Support Forward and Backward Compatibility
6:50 Where to use JSON?
7:03 MicroService to MicroService - Binary Encodings Are Better
8:10 Demo Comparing JSON Encoding with Google Protobuf
10:09 Comparison of HTTP Response Times - JSON vs Google Protobuf
12:12 How Exactly Google Protobuf is more efficient than JSON encodings?


𝗜𝗡𝗧𝗘𝗥𝗩𝗜𝗘𝗪 𝗣𝗥𝗘𝗣 𝗣𝗥𝗢𝗗𝗨𝗖𝗧𝗦
✅ 𝗘𝗱𝘂𝗰𝗮𝘁𝗶𝘃𝗲.𝗶𝗼 [10% OFF for First 90 Users] 👉🏻https://educative.io/rachit
✅ 𝗦𝘆𝘀𝘁𝗲𝗺 𝗗𝗲𝘀𝗶𝗴𝗻 [Discount for Indian audience] 👉🏻https://bit.ly/design-rachit
✅ 𝗣𝗿𝗼𝗴𝗿𝗮𝗺𝗺𝗶𝗻𝗴 𝗕𝗼𝗼𝗸𝘀 [Amazon Affiliate] 👉🏻https://amazon.in/shop/rachitjain

SUBSCRIBE AND HIT BELL ICON TO CHECK MORE OF MY CONTENT
http://youtube.com/RachitJain?sub_confirmation=1


𝗦𝗢𝗖𝗜𝗔𝗟 𝗣𝗥𝗢𝗙𝗜𝗟𝗘𝗦
✅ Portfolio Website - https://rachitiitr.com
✅ Instagram - https://www.instagram.com/rachitiitr
✅ LinkedIn - https://linkedin.com/in/rachitiitr
✅ Twitter - https://twitter.com/rachitiitr
✅ Github - https://github.com/rachitiitr/DataStructures-Algorithms
✅ Facebook - https://fb.me/AlgorithmsWithRachitJain


𝗜𝗠𝗣𝗢𝗥𝗧𝗔𝗡𝗧 𝗣𝗟𝗔𝗬𝗟𝗜𝗦𝗧𝗦
✅ 𝗖𝗼𝗱𝗶𝗻𝗴 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗟𝗲𝗰𝘁𝘂𝗿𝗲𝘀 👉🏻https://www.youtube.com/watch?v=J1JZjhdr3Oo&list=PLfBJlB6T2eOshO8-LYaMt-Pes1IFojJ_l
✅ 𝗚𝗿𝗮𝗽𝗵 𝗧𝗵𝗲𝗼𝗿𝘆 𝗣𝗹𝗮𝘆𝗹𝗶𝘀𝘁 👉🏻https://www.youtube.com/watch?v=xyJxCjweLKE&list=PLfBJlB6T2eOu3dTPKzvAf2axlmQUXGY91
✅ 𝗖++ 𝗦𝗧𝗟 𝗣𝗹𝗮𝘆𝗹𝗶𝘀𝘁 👉🏻https://www.youtube.com/watch?v=g-1Cn3ccwXY&list=PLfBJlB6T2eOvyt21CIX_PMmhOgWHiFVab
✅ 𝗠𝘆 𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝗹 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲𝘀 👉🏻https://www.youtube.com/watch?v=0Vmtmqa9Og0&list=PLfBJlB6T2eOuUAof03sQFlUcrk_Bc7jOA
✅𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝘃𝗶𝘁𝘆 𝗧𝗶𝗽𝘀 𝗣𝗹𝗮𝘆𝗹𝗶𝘀𝘁 👉🏻https://www.youtube.com/watch?v=0Vmtmqa9Og0&list=PLfBJlB6T2eOuUAof03sQFlUcrk_Bc7jOA
✅𝗟𝗶𝗳𝗲 𝗟𝗲𝘀𝘀𝗼𝗻𝘀 & 𝗠𝗲𝗻𝘁𝗼𝗿𝘀𝗵𝗶𝗽 👉🏻https://www.youtube.com/watch?v=9SGM0EqyUc0&list=PLfBJlB6T2eOv8klCHHNVLOmTOIxKu-xzn



𝗣𝗥𝗢𝗚𝗥𝗔𝗠𝗠𝗜𝗡𝗚 𝗣𝗥𝗢𝗙𝗜𝗟𝗘𝗦
✅ Github ► https://github.com/rachitiitr/DataStructures-Algorithms
✅ Programming Blog ► http://rachitiitr.blogspot.com
✅ CodeForces ► http://www.codeforces.com/profile/rachitjain
✅ CodeChef ► http://www.codechef.com/users/rachitiitr