Global monopoly of technology

in the afternoon.

Luo Sheng is still staying at the Bluestar Technology Group. He was going to host a technical meeting at the Blue Coast Laboratory this afternoon. However, after learning that the National Railway Group had come to seek technical support, he temporarily modified his work arrangements. Choose priority to meet Vice President Liu.

Mainly because the New Year’s Eve is approaching, people all over the country, especially migrant workers who go out to work, are rushing home for the New Year. This group is very difficult. Luo Sheng feels that he can do whatever he can for them and do nothing else. It is also a very meaningful thing for them to buy a ticket to go home for the New Year reunion more easily.

What's more, a big customer can be gained.

...

At this moment, in a reception room of the company, Luo Sheng and Vice President Liu and other seven or eight members of the National Railway Group were together.

After a few brief greetings, Vice President Liu found the entry point for the key topic and said: "The development cost of the first phase of the 12306 ticketing system totals 300 million yuan, including hardware. This system needs to be combined with the information database of the Ministry of Railways. The combination of telephone ticketing, window ticketing and other related data. 12306 is not a simple online transaction system, but the largest bulk material freight system in the country and the world."

After a while, an accompanying technical expert also said: "The technology of 12306 is indeed very difficult to solve. The existing distributed database, cache, and load balancing technology can no longer meet the demand."

Speaking of this, the technical expert looked at Luo Sheng and added: "You know, 12306 has nearly 30 billion visits a day during the Spring Festival. Such a large number of train ticket queries is an important factor affecting the performance of 12306. One of the reasons is that it has a high probability of accounting for more than 90% of the visits. What's more difficult is that there is a world of difference between peak and valley queries, and there is almost no way to strike a good balance between cost and concurrency."

Luo Sheng, who was listening, nodded silently. After the deceased had finished speaking, he asked: "How did you solve the cooperation with Ali?"

The technical expert said frankly: "The current approach is to control the flow of several key entrances to ensure the availability of the system, but it will affect the user experience. The website is still down and cannot be opened. It has already been criticized."

Speaking of the technical plane, he shook his head with a wry smile.

Vice President Liu also said: "Ali provides free technical support to 12306, but the technology is still not hard enough. We now put our hope on Nebula."

Luo Sheng couldn't help but smile, and said straightforwardly: "Mr. Liu, first of all, I can definitely give you a reply. The Nebulas system is completely okay. At the same time, I can guarantee that within a week, I can build a complete set of software for 12306. A complete solution to the hardware, but this solution will not be supported for free like Ali."

President Liu was dumb and smiled, and said, "I would like to hear the details."

Luo Sheng thought about it for a moment, and quickly organized the language and said: "If Nebulas supports the 12306 system, put 12306 website query access on Nebula. Generally speaking, query is the most visited part of most ticketing systems. I just listened to you. Experts of said, about 90%. In the transaction-related process, query requests will be submitted many times, not to mention the additional workload brought about by the emergence of a large number of ticket-checking software, all of which leave the remaining votes The query system has become the focus of pressure on the entire system."

"My solution is that after Nebulas intervenes, it separates the remaining ticket query module from the existing 12306 system and has the ability to deploy independently; deploy a set of remaining ticket query system on the cloud independently, so that both 12306 and the cloud are available. A set of remaining ticket query system will be more flexible in scheduling. At present, the scale of Nebulas cluster servers has reached more than 50,000."

Several technical experts from the railway group present at the scene also secretly smacked their lips. No wonder Ali's "Flying" system can't help being beaten. The former barely managed to break 1,000, while the latter has reached a scale of 50,000. No wonder Luo Sheng has such a big deal. confidence.

Next, Luo Sheng mainly discussed with several technical experts of the railway group.

"This morning, when I heard the news, I briefly studied the server architecture of '12306'." Luo Sheng said to everyone in an orderly manner: "The majority of visitors are spraying, but I know that as soon as the 12306 service goes online, it will bear it. With a QPS that cannot be surpassed by any spike system in the world, millions of concurrency cannot be normal."

Several key technical personnel of the railway group present were moved and understood long live.

Leaders who do not understand technology have the most difficulty communicating and feel that they are not doing their best.

Luo Sheng's words are still very important.

After a while, Luo Sheng added: "The high-concurrency system architecture should be deployed in a distributed cluster. The upper layer of the service has layer-by-layer load balancing, and various disaster tolerance methods are provided. The so-called disaster tolerance means are dual fire engine rooms and node fault tolerance. , Server disaster recovery, etc. To ensure the high availability of the system, traffic will also be balanced to different servers according to different load capacities and configuration strategies."

"Even so, the QPS that a single machine in a cluster can withstand is very high, so how to optimize the performance of a single machine to the extreme? To solve this problem, we must first understand one thing: usually the booking system has to deal with generating orders and reducing In the three basic stages of inventory deduction and user payment, what the system has to do is to ensure that the train tickets are not oversold, sold a lot, and every ticket sold must be paid to be effective. Concurrency."

Several technical experts from the railway group nodded in agreement. The technical pain point lies here.

Luo Sheng continued: “Place an order to reduce inventory. When a user’s concurrent request arrives at the server, the order is created first, and then the inventory is deducted, waiting for the user to pay. This order is the first solution we most people think of, and in this case, It can guarantee that the order will not be oversold, but it will also cause some problems. The first is that in the case of extreme concurrency, the details of any memory operation are crucial to affecting performance, especially the logic like creating an order, which basically needs to be stored If you go to a disk database, you can imagine the pressure on the database. 12306 should be the Oracle database. Don't spend the money, put it on my nebula."

"The second is that if the user has maliciously placed an order, only placing an order without paying, the inventory will be reduced, and a lot of orders will be sold less. Although the server can limit the number of IP and the user's purchase order, this is not really true. A good way."

"Then it is to pay and reduce inventory. If you wait for the user to pay for the order and then reduce the inventory, the first feeling is that you will not sell less. But this is a taboo of the concurrent architecture, because in the case of extreme concurrency, users may create many orders When the inventory is reduced to zero, many users will find that the grabbed orders cannot be paid. This is the so-called "oversold", and concurrent operation of database disk IO cannot be avoided."

"The last is withholding inventory. From the considerations of the above two options, it can be concluded that as long as an order is created, database IO must be frequently operated. So is there a solution that does not require direct operation of database IO? The answer is yes , Is to withhold inventory, first deduct the inventory to ensure that it is not oversold, and then generate user orders asynchronously, so that the response to users will be much faster."

"Then how to guarantee a lot of sales? What if the user does not pay after the order is received? The order should be valid. For example, if the user does not pay within five minutes, the order will expire and the new inventory will be added. The generation of the order is asynchronous , It should be placed in the instant consumption queue for processing..."

Vice President Liu heard what was in the clouds, but he found that Luo Sheng and the several technical experts he brought with him had become more and more enthusiastic, and he seemed to have come up with a message.

Find the right person!

At this time, Luo Sheng turned on the big screen on the wall in the room, and he also took a laptop computer to turn on, signaled everyone to look at the main projection screen, and said while operating the computer:

"Go language is originally designed for concurrency. I will use Go language to show you the specific process of single-machine ticket grabbing and my optimized solution."

"The init function in the Go package is executed before the main function, and some preparatory work is also done at this stage. The preparatory work that the system needs to do is to initialize the local inventory, initialize the remote redis to store the hash key value of the unified inventory, and initialize the redis. Link pool."

"In addition, you need to initialize an int type chan with a size of 1. The purpose is to realize the function of distributed locks. You can also directly use read-write locks or use redis to avoid resource competition, but using Channel is more efficient. This is the Go language. The philosophy is that you don’t need to communicate through shared memory, but share memory through communication. The Redis library uses redigo, and the following is the code:

LocalSpike package structure definition

packagelocalSpike

typeLocalSpikestruct{

LocalInStockint64

LocalSalesVolumeint64

}

…

RemoteSpike's definition of hash structure and redis connection pool

packageremoteSpike

Remote order storage key value

typeRemoteSpikeKeysstruct{

...

...

Unknowingly, the time has passed for an hour and a half, anyway, Deputy Liu always doesn't understand, and the code looks dizzy.

The technical issues are finally discussed.

The technical experts of the Railway Group looked at Vice President Liu and nodded his head with a certain look. Vice President Liu immediately understood, and decisively turned to look at Luo Sheng and asked: "Mr. Luo, if you provide such a complete set What is the quotation for the perfect solution?"

Luo Sheng said bluntly: "One billion yuan is paid every year."

When Vice President Liu heard this number, he almost didn't have a mouthful of blood, and the budget for the first phase of the 12306 ticketing system was only 300 million yuan. Such an offer is simply too expensive to accept.

...