Chapter 286 - 165: Forget It
The background noise on the other end of the line was a chaotic mess.
Someone was shouting for the admin at the top of their lungs, "Add ten bucks to machine three in Zone C!" Someone else was slamming their desk and cursing, "What the hell, man? You flashed me!" The cacophony was interspersed with a string of coughs and the CLICK of lighters.
Chen Zhuo held the landline receiver a little further from his ear.
"Hold on a sec."
Chu Ge shouted from the other end, followed by the scraping sound of a chair being dragged. The noise became a bit quieter, as if he’d moved to a corner.
"Okay, go ahead. I found a pen."
Chu Ge clamped the receiver between his shoulder and ear, a candy stick clenched between his teeth. His right hand rummaged through a tangled mess of keyboard cables until it emerged with a ballpoint pen.
There was no paper on the desk. He glanced around, grabbed a leftover instant noodle cup, and tore off the cardboard lid, which was stained with a few drops of red chili oil. He flipped it over.
The back of the lid was slightly damp, but he could still write on it.
Chen Zhuo stood by the living room window, gazing at the thin layer of frost on the glass. He reached out and drew a line through it with his finger.
"How did they build the database in the Capital City?" Chen Zhuo asked.
"Relational. A typical tree structure."
Chu Ge bit down hard on his candy a couple of times.
"They want to build a ’six degrees of separation’ social network. The underlying logic is simple. If I’m A and I know B, the database creates an edge between us. Querying my friends is fast. Querying my friends’ friends... it can just about handle it."
Chu Ge’s voice was hoarse, the kind that comes from pulling an all-nighter.
"But as soon as you try to query third- or fourth-degree connections, if the concurrency goes over two hundred users, the database has to perform endless nested joins. A table with a few hundred thousand users instantly explodes into tens of millions of redundant data rows."
"The guy in the Capital City said they just ran a stress test. Two hundred virtual accounts all clicked the ’People You May Know’ feature at the same time. The server fans started spinning like a helicopter, the CPU shot straight to one hundred percent, and three seconds later, the database deadlocked."
Chu Ge sighed.
"So now it’s an impasse. To make it fast, you have to pre-calculate all the relationships and store them in a cache. But that requires a massive amount of memory, and they can’t afford that many servers. If you don’t pre-calculate and just query in real-time, the database crashes the moment a few people try to access it simultaneously."
After listening, Chen Zhuo didn’t speak immediately.
He walked over to the sofa and sat down, picked up a peanut in its shell from the fruit bowl, and gently cracked it open in his hand.
"You computer science types... are you always so rigid in your thinking?"
Chen Zhuo rubbed the papery skin off the peanut, a hint of amusement in his voice.
"What do you mean?"
Chu Ge was taken aback for a moment.
"This is a pure graph theory problem. Why are you trying to solve it like you’re taking a census?"
Chen Zhuo tossed the peanut into his mouth.
"Ditch your nested queries. Think of all the users as points in a high-dimensional space. The relationships between users are the lines connecting them."
Chu Ge didn’t say a word, his pen tip hovering over the noodle cup lid.
"Build an adjacency matrix."
Chen Zhuo’s voice was steady, as if he were explaining a freshman-level calculus homework problem to someone in a university study hall.
"Let’s say you have 100,000 users. That’s a 100,000 by 100,000 matrix, let’s call it A. If user i and user j are friends, the value at position (i, j) in the matrix is 1. Otherwise, it’s 0."
"Wait a second."
Chu Ge cut him off.
"One hundred thousand times one hundred thousand... that’s ten billion data points. Have you calculated how much memory that would take? The server would blow up on the spot."
"Chu Ge."
Chen Zhuo chuckled softly on his end of the line.
"In real life, do you have a hundred thousand friends?"
"No. I don’t even have ten."
Chu Ge answered honestly.
"No normal person does,"
Chen Zhuo said.
"So, ninety-nine point nine-nine percent of the values in this matrix are 0. It’s an extremely sparse matrix."
𝘧𝑟𝑒𝑒𝘸𝘦𝘣𝑛𝑜𝘷𝑒𝓁.𝘤𝘰𝓂
Chu Ge’s fingers, holding the pen, tightened abruptly.
"In memory, you don’t need to store those ten billion points. You only need to store the coordinates of the ones."
Chen Zhuo continued.
"It’s simple coordinate mapping. Next, if you want to find your friends’ friends—your second-degree connections—how would you express that mathematically?"
Chu Ge’s mind raced. The linear algebra concepts he’d learned as a freshman and had mostly forgotten suddenly cut through the fog in his brain like a bolt of lightning.
"Matrix multiplication,"
Chu Ge blurted out.
"A times A. A squared."
"Right."
Chen Zhuo stood up, walked to the coffee table, and poured himself a glass of warm water.
"A squared. The non-zero values in that matrix are your second-degree connections. A cubed gives you your third-degree connections."
Chu Ge stared at the noodle cup lid, his ballpoint pen unconsciously drawing circles on it. His breathing grew faster.
"But the computational load for matrix multiplication is huge, especially for higher-order multiplications. The CPU still won’t be able to handle it."
Chu Ge raised the core issue.
"That’s because you’re being too greedy. You’re trying to calculate the entire social network for everyone all at once."
Chen Zhuo took a sip of water, wetting his throat.
"The server doesn’t need to know who everyone knows. It only needs to answer the request of the currently logged-in user. Therefore, you don’t need to do matrix-by-matrix multiplication."
Chen Zhuo’s voice grew quiet, but to Chu Ge’s ears, it was like a clap of thunder.
"The currently logged-in user isn’t a matrix. It’s a vector."
Chen Zhuo delivered the final piece of the solution.
"A vector that’s a 1 at the user’s own position and 0 everywhere else. You take that one-dimensional vector and multiply it by the sparse matrix. One multiplication, and you get a vector representing their direct friends. Take that result and multiply it by the matrix again, and you get their second-degree friends."