The story of "True® Full Stack Road-DNS Chapter" starts by entering the URL...

The story of "True® Full Stack Road-DNS Chapter" starts by entering the URL...

Preface

It’s been a long time since I wrote a blog. My original title was "XXX from input url to page loading"?

But thinking about it, this is the content that others have chewed up many times, and it is not challenging. Moreover, there are too many places to be optimized in the process of page operation.

Then just dig a hole for yourself. Anyway, I spent a year in the operation and maintenance development department.

This paper attempts from the front and back ends of a plurality of angles or operation and maintenance , to tell from the parsed to optimize the operation of the entire site.

1. Process review

1. A series of events from the input of URL to browser parsing

Many big company interviews like to ask such an interview question. What happened after entering the URL and seeing the page? , Let’s summarize today. Simply put, there are the following processes

  1. Enter the URL in the browser
  2. Domain name resolution ( DNS), find the IP server
  3. Initiate TCPconnections, HTTPthree-way handshake, sending a request Request( )
  4. Server response HTTP (Response)
  5. Browser to download resources html css js images, etc.
  6. Browser parsing code (if the server has gzipcompression, the browser first extract)
  7. Browser rendering to the user

2. Combine the operation page to the close tab page

We perform certain operations after the page is rendered:

  • Button repeat click
  • Scrolling
  • Conditional query retrieval

Let's put all of the above as ==> 8. Interface operation

Also insert before Step 3: Initiate TCP connection :

  • Optimization of concurrent requests allowed by the browser

Let's start with DNS resolution...

2. DNS resolution process

To Chromethe browser, for example:

  1. The Chrome browser will first search the browser's own DNS cache. (The cache time is relatively short, only 1 minute by default, and can only accommodate 1000 caches)

Note: chrome://net-internals/#dnsto view Chrometheir own cache)

  1. If the browser cache itself there is no corresponding entry is found, then Chromesearches for the operating system itself DNS cache
  • Windows- In Windowsview of the DNScache entry process is very simple. Simply open a command prompt and enter the following command: ipconfig/displaydns.
  • Mac- in Macview the DNScache entry process is slightly different. You need to open the console application first, select the device from the left sidebar, and then type: any:mdnsresponderenter the search bar. Next, open the command line and enter the following command: sudo log config --mode "private_data:on" sudo killall -INFO mDNSResponder Then, return to the console application and view the list of cached DNS records. For example, the following screenshot shows the wx.qlogo.cncached CNAMErecord.
  1. If you did not find the DNS caching system, then try to read the hostsfile. Check if there is an IP address corresponding to the domain name, if there is, the resolution is successful.
    • Note: If you are Windowslocated C:\Windows\System32\drivers\etc, Macit is /etc/hosts.
    • This operating system-level domain name resolution often exploited by malicious, you modify the hostscontents of the documents to the DNS his designated ipon the address, resulting in so-called domain name hijacking, so the hostsfile settings become read-only mode , To prevent malicious tampering.
  2. If the hostscorresponding entry does not find the file, the browser will initiate a DNSsystem call requesting the local domain name server localDNS( ) LDNSto resolve the domain name.
  • By UDPagreement to DNSport 53 initiated the request, the request is a request recursive, that is, the operator's DNSservers have to provide us with the domain name IPaddress)
  • The local domain name server ( LDNS) will be requested for the first time to resolve this domain name. This service is generally located in a corner of your city, not very far away, and its performance is very good. Generally, it will cache the domain name resolution results, about 80%. The resolution of% domain names is over here.
  • If the local domain name resolution server does not have a record of the domain name, it will start recursive + iterative resolution

Up to this point, all DNS resolutions that the browser can do have been completed, and the next step is related to the server. You can ignore what you don't want to see.

  1. If you localDNSstill have not hit directly to the RootServerdomain name server to resolve the request.
  2. The root domain name server returns a queried primary domain name server ( gTLDServer) address to the local domain name server . gTLDIt is an international top-level domain name server, such as,, .com. .cn, orgetc., there are only about 13 in the world.
  3. A local domain name server localDNSthen returns the step up gTLDserver sends a request.
  4. Accept the request of the gTLDserver finds and returns the domain corresponding to the NameServeraddress of the DNS server, this NameServeris usually the user to register a domain server, such as a user in the domain name application for a domain name service provider, then the task of the DNS domain name provider Server to complete.
  5. NameServerDomain names and IP mapping table name server queries stored, have been the target IP address based on the domain name under normal circumstances, together with a TTL value is returned to the DNSServerdomain name server.
  6. Returns corresponding to the domain IPand TTLvalues LDNScached domain name and IPa correspondence relationship, the cache time TTLvalue control.
  7. The result of the resolution is returned to the user, and the user caches the local system cache according to the TTL value, and the domain name resolution process ends.

Note: In the actual DNS resolution process, possibly more than the 11 steps (step 1 in fact negligible.), As NameServerthere may be a lot of class, or have a GTMto load balance control, which are likely to affect the name resolution process .

If you don’t want to read the text, you can look at the picture:

3. DNS optimization

First of all, it needs to be clear: DNS cache has multiple levels of cache. If sorted by the distance from the browser, there are the following:

  • Browser cache
  • System cache
  • Router cache
  • IPSServer cache
  • Root DNS cache
  • Top-level domain name server cache
  • The main domain name server cache.

If you go through so many steps every time, is it too time-consuming? How to reduce the number of steps in this process? It needs DNSoptimized. In the front-end optimization and DNSrelated to two things:

  • Reduce DNSthe number of requests
  • DNSPre-analysis

DNSAs the basic protocol of the Internet, its resolution speed seems to be easily overlooked by website optimizers. Now most of the new browser has been optimized for DNS resolution, typical of a DNSparsing takes 20-120milliseconds, reduce the number of DNS resolution time and is a good way to optimize. There is no longer tell, talk focused on DNSthe pre-resolving it.

3.1 Front end: DNS prefetch

DNS prefetchThe domain name with this attribute is resolved in the background without the user clicking on the link, while domain name resolution and content loading are serial network operations, so this method can reduce user waiting time and improve user experience.

By default, the browser will pre-fetch domain names that are not in the same domain as the current domain name (the domain name of the web page being browsed) on the page, and cache the result, which is implicitDNSPrefetch .

If you want to pre-fetch a domain that does not appear on the page, then you must use the displayed one DNSPrefetch.

Its usage is as simple as linkadding a corresponding label properties:

/* This is used to tell the browser to do DNS pre-resolution on the current page*/
<meta http-equiv="x-dns-prefetch-control" content="on"/> 
<link rel="dns-prefetch" href="//example.com">
  • If you need to access a large number of resources of different domain names in your page, you can use this technology to speed up the acquisition of resources, so as to obtain a better user experience.
  • It should be noted that DNSalthough pre-analysis is good, it cannot be abused. If DNS pre-resolution is repeated for multiple pages, the number of DNS queries will increase.

At present, many large sites have also applied this optimization, such as:

Taobao:

Jingdong:

If you need to prohibit implicit DNSPrefetch, you can use the following tags:

<meta http-equiv="x-dns-prefetch-control" content="off">

3.2 Backend & Operation and Maintenance: CDNandHTTPDNS

In fact, there are three optimizations that can be done for back-end & operation and maintenance:

  • CDN
  • HTTPDNS
  • ~~ DNSLoad Balancing~~

However, for slightly larger websites, DNS load balancing is basically abandoned, and there are too many shortcomings. Those who are interested can search and understand by themselves.

1. CDNthe DNScycle

CDN, The full name is ContentDeliveryNetwork, that is , the content distribution network. Its purpose is through the existing Internetincrease in new layer of CACHE(buffer) layer, the content of the site to publish the network node "edge" closest to the user, so that users can go to obtain the desired content, improve user access to the site Response speed.

DNSCycle: When the authority DNSfinds a plurality of name mapping IP, it uses IPround-robin fashion to be IPequally distributed to the plurality of DNSrequest, so as to achieve load balancing.

Why is it needed CDN?

  • Since the DNSaverage distribution of the cycle, can not be optimized according to the load distribution on different servers, even if one server goes down, DNSyou can not be kept informed of the situation to the server IPallocated out, will result inaccessible.
  • Therefore, the authority DNSbetween the server and add a CDNlayer becomes very necessary.
  • CDN On the basis of the ability to dispatch and distribute servers, it can synchronize the running status of the servers, and then appropriately adjust the scheduling strategy in time according to the situation, so that the load balancing ability is greatly improved.

CDNbenefit:

  • Solve the "first mile" problem on the server side.
  • Alleviate or even eliminate the impact of bottlenecks in interconnection between different operators.
  • Reduce the pressure on export bandwidth of various provinces.
  • Relieved the pressure on the backbone network.
  • Optimized the distribution of hot content on the Internet.

CDNAccess steps:

(1) Before deploying CDN application:

Network request path:

  • Request: Local network (local area network)——"Carrier network——"Application server room
  • Response: Application server computer room——"Operator network——"Local network (LAN)

Without considering the complex network, in response to a request from the need to go through 3nodes, 6steps to complete a user access operation.

(2) After deploying the CDN application:

Network path:

  • Request: Local Network (Local Area Network)-"Operator Network
  • Response: Operator's Network-"Local Network (LAN)

Without considering the complex network, in response to a request from the need to go through 2nodes, 2steps to complete a user access operation.

And not to deploy CDNcompared to services, reduce 1nodes, 4access steps. Greatly improve the response speed of the system.

The following are the steps to combine specific network operation and maintenance:

step1: The user initiates a request to localDNS to query the IP address corresponding to the input domain name (if there is a cache, return directly, otherwise go to rootDNS for query);

step2: localDNS iteratively queries rootDNS, iterating step by step, rootDNS=>top DNS=>authority DNS;

step3: After obtaining the authority DNS, localDNS initiates a domain name resolution request to the authority DNS;

step4: Authority DNS usually will CNAME the domain name
[If there is a CNAME, resolve the CDN service corresponding to the CNAME, otherwise it will default to a normal request, and directly return the resolved IP] to another domain name, this domain name will eventually be directed to the intelligent DNS load balancing system in the CDN network;

step5: DNS load balancing system returns the most suitable CDN node IP address to localDNS through some intelligent algorithms;

step6: localDNS will return the obtained IP address to the user;

step7: After the user gets the IP address of the node, he initiates an access request to the node;

Step8: The CDN node returns the requested file. If the requested file does not exist in the node, it will return to the source site to obtain the file, and then return it to the user.

2 HTTPDNS: solve the DNShostage:

From: Also talk about HTTPS-HTTPDNS + HTTPS

In the usual impression, very often I feel the site to complete HTTPSagreement VANS. actually not:

  • Despite the use of HTTPStechnology, part of the evil of operators, still using DNSpolluting technologies, so that the domain name pointing to their own servers.
  • These servers do not deploy SSLservice (even if deployed, will trigger SSLa certificate Commonnamemismatch alarm), resulting in 443 direct port is rejected.

Operators are unscrupulous in order to earn advertising money and save inter-network settlement. Hijacking means they are commonly used by ISPproviding a DNSfake domain name. That there is no way to solve the DNShijack it?

The industry has a set of solutions for such scenarios, namely HTTPDNS:

HTTPDNSUsing the HTTPprotocol name resolution, based on replacing the existing UDPDNS protocol, domain name resolution request directly to the HTTPDNSserver, thereby bypassing the operator LocalDNScan avoid 免LocalDNSthe domain name hijacking problems caused by inaccurate and scheduling problems.

HTTPDNSThe principle is very simple, to DNSsuch a protocol could be hijacked, using the converted HTTPprotocol request Domain<-> IPmapping. Correct IPafter, Clientto assemble their own HTTPprotocol, so as to avoid ISPtampering with the data.

Tencent first company to deliver HttpDNSa cloud service provider services, there are two articles published four years apart, which reveals very detailed technical details:

To be continued...

Next to speak about the contents HTTP优化of two general direction ::

  • Reduce the number of requests.
  • Reduce the time spent on a single request.

The corresponding contents are:

  • Optimization of concurrent requests allowed by the browser, nginxconfiguration/domain name divergence and convergence.
  • The compression and merging of resources are webpack/Gziprelated.
  • There are other content dictated by interest...

Author Nuggets Article Collection

If you need to repost it to the official account, just call me to add the whitelist.

  • "True® Full Stack Road" Back-end guide for web front-end development
  • "Vue Practice" A Vue CLI plug-in in 5 minutes
  • "Vue practice" arm your front-end project
  • "Intermediate and advanced front-end interview" JavaScript handwritten code invincible cheats
  • ``Learn from the source code'' Vue question answers that interviewers don't know
  • "Learn from the source code" JS Sao operation in Vue source code
  • "Learn from the source code" thoroughly understand the Vue option Props
  • The correct posture of the "Vue practice" project to upgrade vue-cli3
  • Why can't you understand the JavaScript scope chain?
Reference: https://cloud.tencent.com/developer/article/1488438 The story of "True® Full Stack Road-DNS Chapter" starts by entering the URL...-Cloud + Community-Tencent Cloud