TechFlix v0.7

Stay updated with the latest tech, engineering and research content

Note: Blog summaries are generated by AI and may sometimes contain errors, irrelevant words or incomplete sentences

Native Frame Rate Playback

Netflix

Algorithms

Published: 5TH June, 2023

Author: Netflix Technology Blog

Summary:
  • HDMI QMS is positioned to be the ideal solution to address the problem we are presenting.
  • Unfortunately, at present, this technology is relatively new and adoption into source and sink devices will take time.
  • Every source device that integrates the Netflix application is required to let the application know if it and the connected sink device have the ability to send and receive video content at its native frame rate.
  • A user might have selected not to do this via the source device system settings e.g.
  • “Match Content Frame Rate” set to “Never” or they might have indicated a To handle these issues we enable the Native Frame Rate playback experience only when the user selects a title and watches it in full screen with minimal graphical UI elements.
  • blended together on the source device.
Check it out!
Tweet it 👆

A New Set of APIs for Amazon SQS Dead-Letter Queue Redrive

Amazon Web Services

Cloud
Backend
Algorithms

Published: 8TH June, 2023

Author: Sébastien Stormacq

Summary:
  • AWS launchedAmazon SQSonJuly 12, 2006.
  • Amazon SQS is a highly scalable, reliable, and elastic queuing service that “just works.” On macOS I enterbrew upgrade awscli.
  • latest version of the AWS CLI.
  • I first create two queues.
  • One is the dead-letter queue, and the other is my application queue.
  • I enter a redrive policy: post messages in the DLQ after three delivery attempts.
  • I consume the message, but I don’t delete it from the queue.
  • This simulates "TaskHandle": "eyJ0YXNrSWQiOiI4ZGJmNjBiMy00MmUwLTQzYTYtYjg4Zi1iMTZjYWRjY2FkNmEiLCJzb3VyY2VBcm4i OiJhcm.
Check it out!
Tweet it 👆

Identifying Green Vehicles for a Zero-Emission Future

Uber

Performance
Backend
Data Science
Algorithms

Published: 8TH June, 2023

Author: Uber

Summary:
  • Uber has made a public commitment to phase out carbon emissions in the United States, Canada, and Europe by 2030, and worldwide by 2040.
  • In some markets, if you request Uber Green, you’ll get a ride in either a hybrid or an electric vehicle.
  • In others, Uber Green is 100% electric.
  • Uber’s onboarding processes vary significantly by region.
  • The most accurate vehicle data comes via vehicle identification numbers (VINs) Not all registration docs in all jurisdictions provide VINs, nor contain enough information to accurately identify if a vehicle is zero- or low-emissions.
  • Uber has millions of vehicles registered on the platform, and hundreds of thousands more onboarding every month.
  • Despite how they may appear in the real world, in Uber’s databases vehicles are not static entities.
  • Their data are subject to change as more docs are uploaded or processed.
  • Resiliency at scale becomes key to a working solution.
  • The main intent is to be able to identify vehicle types as soon as they’re onboarded, and when any updates (VIN number, Make or Model info, etc.) are made to the vehicle information.
  • The diagram below presents an overview of the various system components that comprise the Green Vehicle Identification (GVID) architecture.
  • VIN Store is an internal service that captures and standardizes VIN responses from various vendors that Uber partners with.
  • It follows an Adapter design pattern to make it easy to onboard new vendors and capture additional attributes from existing vendors.
  • The identification logic is also decoupled from Operations workflows that add appropriate labels to vehicles.
  • Previously, the labeling process to encode region-specific business rules required careful, manual effort.
  • With the new architecture, the identification process is triggered immediately upon the processing of the vehicle document, resulting in the process completing in a matter of minutes.
  • The automation of labeling of vehicles to make them eligible for low-emission products and incentives has led to elimination of manual toil.
  • Multiple hours of effort was being spent every week to run the workflows, monitor them and also resolve any issues resulting due to failures.
  • The architecture enables us to have a single source of truth that contains vehicle specs and engine information.
  • Uber developed a scalable architecture that can leverage multiple data sources to determine a vehicle’s engine and fuel type.
  • This system now powers our low-emission ride options in major markets.
Check it out!
Tweet it 👆

LinkedIn Bug Bounty Program - One Year Anniversary of Public Launch

LinkedIn

Mobile
Algorithms

Published: 2ND June, 2023

Author: Rohit Pitke

Summary:
  • LinkedIn launched its bug bounty program publicly on May 18th, 2022.
  • The program experienced a significant increase in the number of researchers submitting security reports.
  • The number of active participants of our public program is over 5 times higher than our private program.
  • This influx of talented researchers has enriched the program.
  • The public bug bounty program helped us improve processes and strengthen defenses across our products and platforms.
  • Here are some key takeaways from our journey so far.
  • Invest in automation wherever possible throughout the lifecycle of a vulnerability.
  • Some of the top researchers on the program had this to say regarding their experience on LinkedIn’s bug bounty program.
  • The complexity and interconnectivity of LinkedIn's product ecosystem presents an engaging and thought-provoking challenge for security testing.
Check it out!
Tweet it 👆

Cloudflare Area 1 earns SOC 2 report

Cloudflare

Published: 8TH June, 2023

Author: Samuel Vieira

Summary:
  • Cloudflare Area 1’s SOC 2 Type II report covers a 3 month period from 1 January 2023 to 31 March 2023.
  • Our auditors assessed the operating effectiveness of the 70 controls we’ve implemented to meet theTrust Services Criteria.
Check it out!
Tweet it 👆

Understand the impact of Waiting Room settings with Waiting Room Analytics

Cloudflare

Published: 7TH June, 2023

Author: Arielle Olache

Summary:
  • The requests to the website behind a Waiting Room go to a Cloudflare data center that is close to theirlocation.
  • Our workers from around the world record values in the histogram which is compressed and sent to the data center Durable Object periodically.
  • The resulting histogram is the final data structure which is used for statistical analysis.
  • Waiting Room Analytics samples the data in order to effectively run large queries.
  • ABR adaptively chooses the resolution of data based on the query.
  • As of now, the analytics data is available in the Clickhouse table for 30 days.
  • The weighted average is computed to provide a more accurate representation.
  • This approach takes into account the distribution and significance of different data points.
  • For example, to evaluate the weighted average for time spent on origin, the value oftotal active usersis used as a weight.
  • The query is done on the zone level.
  • to calculate the average, maximum and minimum oftotal active users, estimated wait time, total queued users and session durationevery fifteen minutes.
  • The response is obtained in a JSON format.
Check it out!
Tweet it 👆

Celebrating Pride 2023 at Confluent

Confluent

Web

Published: 8TH June, 2023

Author: Harini Rajendran

Summary:
  • The month of June in the U.S.
  • is dedicated to celebrating the LGBTQIA+ community.
  • Pride celebrations take place during other months like February (Australia), August (Canada) and September (Taiwan) Confluent’s Queerfluent ERG is a thriving place for employees who belong to the community.
  • Being both gay and butch, as a trans woman, the signs were a little less obvious.
  • Of the more than 2,000 people at Accenture Australia at the time, I was the first to come out as transgender.
  • HR asked me to hold off on coming out for a couple of weeks while they figured out what to do.
  • Matt Gollings, an older, ex The Confluent team’s biggest impact on my queer identity has, and always will be, their incredible ability to normalize my experience.
  • This empowers me to grow, not only from a work perspective but from a very personal perspective as well.
  • For me personally, as a queer community member, I feel that I’m part of the group, able to be myself Harini Rajendran is a senior software engineer at Confluent.
  • She came out to her team after coming out as queer.
  • She says the team has been incredibly welcoming.
Check it out!
Tweet it 👆

GPT-4 + Streaming Data = Real-Time Generative AI

Confluent

Web
AI

Published: 8TH June, 2023

Author: Michael Drogalis

Summary:
  • ChatGPT can't help here because it doesn’t know the answer to these questions.
  • The fundamental obstacle is that you, the airline company, need to safely provide timely data from your internal data stores.
  • Large language models have changed the relationship between data engineering and model creation.
  • ChatGPT is a neural network trained on text from the internet.
  • By training on such a large corpus of data, GPT has been able to infer an astonishing amount about how to converse like a human.
  • ChatGPT has something called a context window, which is like a form of working memory.
  • ChatGPT can’t tell your customer if their flight was delayed or if they can upgrade to first class.
  • When the number of tokens exceeds the window size, the oldest tokens get dropped off the back.
  • The answer is to modify GPT and work with it directly.
  • This approach is a lot easier to understand, less error-prone, and more suitable for situations that require factual answers.
  • While it might look like a hack, this is exactly the approach being taken by some of the best-known AI products like Copilot.
  • Embedding is a way to map things into a “concept space” as vectors of numbers.
  • You can then use fast operations to determine the relatedness of any two concepts.
  • A vector database specializes in organizing and storing this kind of data.
  • Pinecone, Weaviate, Milvus, and Chroma are popular choices.
  • With your policies in a vector database, harvesting the right information becomes a lot simpler.
  • By again using stream processing, we can keep track of how helpful the agent is from moment to moment.
  • We can feed that knowledge back into the application so it can dynamically adjust how it constructs its prompt.
  • Fine-tuning allows you to supply vastly more information to the model once, rather than paying the cost every time a prompt is run.
  • It’s a technique that should be used in conjunction with prompt augmentation, ratherthan something you’d use exclusively.
Check it out!
Tweet it 👆

Examining HTTP/3 usage one year on

Cloudflare

Published: 6TH June, 2023

Author: David Belson

Summary:
  • The real-time benefits of using the newest version of the web’s foundational protocol may not be completely applicable for automated content retrieval.
  • remains minimal to non-existent.
Check it out!
Tweet it 👆

From the Economic Graph to Economic Insights: Building the Infrastructure for Delivering Labor Market Insights from LinkedIn Data

LinkedIn

AI

Published: 2ND June, 2023

Author: Akash Kaura

Summary:
  • LinkedIn’s vision is to create economic opportunity for every member of the global workforce.
  • Since its inception in 2015, the Economic Graph Research and Insights (EGRI) team has worked to make this vision a reality.
  • We have seen incredible interest in the insights that the EGRI team can provide.
  • Over 50 requests for our data and insights per month.
  • LinkedIn's data infrastructure must be able to handle a high volume of requests from a wide range of consumers.
  • LinkedIn’s members rely on the platform to keep their data secure.
  • The EGRI team operates with the following guiding principles as our collective north star.
  • The pipeline for serving our LinkedIn Hiring Rate (LHR) metric is a prime example of our use of tools and operating principles.
  • For LHR1, we need to take in data from across the LinkedIn ecosystem including (but not limited to) data on our members’ profiles.
  • The Asimov team has limited resources, and here, the Accordance principle comes into play.
  • Having a clear understanding of the relative prioritization of metrics and datasets, with the buy-in of the full team, allows us to direct resources to the most critical areas.
  • We’ll continue to onboard our metrics onto our foundational ecosystem.
  • Project Asimov is the brainchild of Casey Weston, Paul Ko, and Rosie Hood.
  • It is based on the ideas of Isaac Asimov, the creator of theimovimovimov.com.
  • Project Asimov was created by Paul Ko and Casey Weston.
Check it out!
Tweet it 👆

AWS Week in Review – Amazon Security Lake Now GA, New Actions on AWS Fault Injection Simulator, and More – June 5, 2023

Amazon Web Services

Cloud
Security

Published: 5TH June, 2023

Author: Veliswa Boya

Summary:
  • Last week I traveled to Cape Town to speak at the.Net Developer User Group.
  • I enjoyed my time there—what an amazing community!Join the group in order to learn about upcoming events.
  • Now onto the AWS updates from last week.
  • AWS Silicon Innovation Day (June 21) A one-day virtual event that will allow you to understand AWS Silicon and how you can use it.
Check it out!
Tweet it 👆

All about the goals: Meet Andrea Fernandes, Swiggy’s Product head for checkout and payment

Swiggy

Published: 6TH June, 2023

Author: SwiggyBytes

Summary:
  • Andrea Fernandes is the Head of Product for Checkout & Payment at Swiggy.
  • She is an avid sports woman and played Hockey at the national level.
  • Her career path that led to Swigy has been filled with tackles, hits, misses and brilliant goals.
  • After her stint at Airtel, Andrea moved to Amazon, where she learnt the true meaning of “customer obsession” “I really liked that concept — there I learnt how to be super-focussed on customers and what they need.
  • That’s what I liked about Swiggy as well and that made me want to join Swigy.
  • Andrea has been with Swiggy for a year and a half and she has been enjoying the experience.
  • “Sports teaches you so much about life.
  • fact that you will get better.
  • And, just go for it!” she says.
Check it out!
Tweet it 👆

Gotchas of Streaming Pipelines: Profiling & Performance Improvements

Lyft

Performance
Coding

Published: 6TH June, 2023

Author: Rakesh Kumar

Summary:
  • We monitor the pipelines on 3 axes: latency, availability, cost, and optimizing each according business requirements.
  • While building a pipeline, do not pre-optimize the pipeline.
  • Consider it to be an iterative process that is followed throughout the lifecycle of the pipeline, and gradually improve performance.
Check it out!
Tweet it 👆

🧑🏻‍💻 xqbuilds

 adityavivek.xq@gmail.com | @xq_is_here undefined

Made with 💖 by XQ. ©2023 XQ Builds