HTTP network programming in Java (1): TCP / SSL web page download

Charzueus 2021-02-23 14:34:53
http network programming java tcp


Catalog

One 、 brief introduction :HTTP Programming

    1、HTTP The system design

    2、HTTP Client working process

    3、HTTP The working process of the server

Two 、 be based on TCP Socket Of HTTP Website download

3、 ... and 、 be based on SSL Socket Of HTTPS Website download

Four 、HTTP Client complete code

5、 ... and 、 Interface integrity code

6、 ... and 、 Last + demonstration


One 、 brief introduction :HTTP Programming

Final review HTTP Network programming , Main learning records HTTP(s) Network programming of protocol , Including the use of TCP Socket Three handshakes HTTP Website download , And use SSL Socket Safe transmission of HTTPs Website download , Complete the programming by case practice , know http(s) The actual working mechanism of !

current HTTP The client is much more complex than the earlier ones , It's not just about downloading and displaying web files , There are many new features : Cross platform display 、 Parameter passing 、 The realization of dynamic web page and user interaction, etc .

1、HTTP The system design

  • Client software (web browser :Chrome、360 The browser etc. )
  • Server software (web The server : Microsoft IIS、Apache Tomcat)

2、HTTP Client working process

  • The client software establishes a connection with the server (TCP Three handshakes of );
  • send out HTTP Header format protocol ;
  • Receiving web files ;
  • Show web page .

3、HTTP The working process of the server

  • Turn on the server software 80 port ;
  • Respond to customer requests 、 complete TCP Connect ;
  • Check the client's HTTP Header format to send the web file requested by the customer ( With dynamic web pages ). 

chart 1 HTTP request - Respond to the complete process

Web download technology is a search engine 、 Web crawler 、 Web page collector or network push services and other related applications in the field of basic technology , Here are two protocols used in daily life (http and https) Web access to download .

Two 、 be based on TCP Socket Of HTTP Website download

about TCP Socket connection process has a very deep understanding of , It has also been used to test communication locally TCP Of Socket Establishing a connection , Empathy , And HTTP Server establishes connection , It's also the use of TCP Information exchange .

Once the connection is established , Need to send HTTP Request header , The server confirms the requester , Turn on communication at both ends , The client can receive web file information , After rendering, the web page will be displayed . Here we realize to receive web page file information first , In the next article, we will realize the function of the browser after rendering the web page .

With www.baidu.com For example , And HTTP After the server establishes the connection , We need to send a web request , That is to say HTTP Request header . Construct the request header as follows :

GET / HTTP/1.1  

HOST: www.baidu.com

Accept: */*

Accept-Language: zh-cn

User-Agent: User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64)

Connection: Keep-Alive

  It needs to be sent strictly in accordance with the format , And commonly used StringBuffer Class toString() Method can complete HTTP The request header is converted to a string , Send consistently to HTTP The server .

StringBuffer msg = new StringBuffer();
msg.append("GET / HTTP/1.1\r\n"+
"HOST: "+domainName+"\r\n"+
"Accept: */*\r\n"+
"Accept-Language: zh-CN\r\n"+
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)\r\n"+
"Connection: Keep-Alive\r\n"
);

 

  Line breaks use \r\n To avoid errors due to coding problems .

After sending the request, if the first message returned by the web information display area is “HTTP/1.1 200 OK”, It means that the visit is normal .

You can see HTTP The server returns a lot of information , This is also the response header , Contains a lot of key information content .

3、 ... and 、 be based on SSL Socket Of HTTPS Website download

Based on the above design TCP Communication transmission HTTP, We try to visit www.sina.com.cn, It turns out that the first line of the response header is HTTP/1.1 302 Moved Temporarily( Site removed ), For safety reasons , Now most of the web The site will give up HTTP Enabling HTTPS, They're all using secure encryption HTTPS agreement , And shut it down HTTP, Only enabled SSL/TLS Of HTTPS A secure connection , By default, this connection uses 443 port . therefore TCP Socket There is no normal way to visit the web page .

It's just that the port was changed to 443 Can it be normal , The answer is as follows .

The reason can also be seen in the front , Need to use SSL/TLS Of HTTPS A secure connection , To establish and HTTPS Server communication , So it needs to be modified Socket type .

It's used here Java Secure socket extension (Java Secure Socket Extension,JSSE), be based on SSL and TLS Agreed Java Web applications provide Java API And reference implementation , The client's SSLSocket Socket .SSLSocket Relative to the client socket I learned before , It's just that the creation method is different ,SSLSocket Object by SSLSocketFactory establish .

Declare member variables in a class and create Socket Connect :

private SSLSocket socket;
private SSLSocketFactory factory;
factory=(SSLSocketFactory)SSLSocketFactory.getDefault();
socket=(SSLSocket)factory.createSocket(ip,Integer.parseInt(port));

 

  Yes SSL Socket Use and TCP identical , It's just that the creation method is different , After a little modification , Can successfully request HTTPS Web page information of the website .

Four 、HTTP Client complete code

Here is given HTTP The complete code of the client ,HTTPS Just change the above SSL Socket.

/*
* HTTPClient.java
* Copyright (c) 2020-12-21
* author : Charzous
* All right reserved.
*/
package chapter08;
import java.io.*;
import java.net.Socket;
public class HTTPClient {
private Socket socket;
private PrintWriter pw;
private BufferedReader br;
/**
* @param ip
* @param port
* @return
* @author Charzous
* @date 2020/12/21 14:52
*
*/
public HTTPClient(String ip, String port) throws IOException{
// Actively initiate a connection to the server , Realization TCP Three handshakes
// If you don't succeed, you throw an error , It's up to the caller to handle the error 
socket =new Socket(ip,Integer.parseInt(port));
// Get the network stream output byte stream address , And encapsulate it into network output character stream 
OutputStream socketOut=socket.getOutputStream();
// Parameters true Indicates automatic flush data 
pw=new PrintWriter(new OutputStreamWriter(socketOut,"utf-8"),true);
// Get the network input byte stream address , And encapsulated into a network input character stream 
InputStream socketIn=socket.getInputStream();
br=new BufferedReader(new InputStreamReader(socketIn,"utf-8"));
}
public void send(String msg) throws InterruptedException {
// Output character stream , from socket Call the underlying system function , Send byte stream through network card 
try {
Thread.sleep(500);
}catch (InterruptedException e){
e.printStackTrace();
}
pw.println(msg);
}
public String receive(){
String msg=null;
try {
// Read information from a stream of network input characters , Only one line of information can be accepted at a time
// When you don't have a line ( No line terminator ), The statement blocks
// Until the conditions are met , The program runs down 
msg=br.readLine();
}catch (IOException e){
e.printStackTrace();
}
return msg;
}
public void close(){
try {
if (socket!=null)
socket.close();
}catch (IOException e){
e.printStackTrace();
}
}
}

 

5、 ... and 、 Interface integrity code

I use a graphical interface to access http and https, Integrate the functions of the above two graphics clients , Make the graphic client can access 443 Of https Content , You can also visit non 443 port ( It's usually 80) Of http Content .

/*
* HTTPAllClientFX.java
* Copyright (c) 2020-12-21
* author : Charzous
* All right reserved.
*/
package chapter08;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.geometry.Insets;
import javafx.geometry.Pos;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.Label;
import javafx.scene.control.TextArea;
import javafx.scene.control.TextField;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
public class HTTPAllClientFX extends Application {
private Button btnExit=new Button(" sign out ");
private Button btnSend = new Button(" Web request ");
// private TextField tfSend=new TextField();// Input information area 
private TextArea taDisplay=new TextArea();// Display area 
private TextField ipAddress=new TextField();// Fill in ip Address 
private TextField tfport=new TextField();// Fill in the port 
private Button btConn=new Button(" Connect ");
private HTTPSClient httpsClient;
private HTTPClient httpClient;
private Thread readThread;
public static void main(String[] args) {
launch(args);
}
@Override
public void start(Stage primaryStage) {
BorderPane mainPane=new BorderPane();
// Connect to the server zone 
HBox hBox1=new HBox();
hBox1.setSpacing(10);
hBox1.setPadding(new Insets(10,20,10,20));
hBox1.setAlignment(Pos.CENTER);
hBox1.getChildren().addAll(new Label(" Web address :"),ipAddress,new Label(" port :"),tfport,btConn);
mainPane.setTop(hBox1);
VBox vBox=new VBox();
vBox.setSpacing(10);
vBox.setPadding(new Insets(10,20,10,20));
vBox.getChildren().addAll(new Label(" Web information display area "),taDisplay);
VBox.setVgrow(taDisplay, Priority.ALWAYS);
mainPane.setCenter(vBox);
HBox hBox=new HBox();
hBox.setSpacing(10);
hBox.setPadding(new Insets(10,20,10,20));
hBox.setAlignment(Pos.CENTER_RIGHT);
hBox.getChildren().addAll(btnSend,btnExit);
mainPane.setBottom(hBox);
Scene scene =new Scene(mainPane,700,500);
primaryStage.setScene(scene);
primaryStage.show();
// Connect button 
btConn.setOnAction(event -> {
String ip=ipAddress.getText().trim();
String port=tfport.getText().trim();
taDisplay.clear();
try {
if (port.equals("443")){
httpsClient = new HTTPSClient(ip, port);
// Successfully connected to the server , Accept the first welcome message from the server 
taDisplay.appendText(" Server connection successful .\n");
readThread = new Thread(()->{
String receiveMsg=null;// Receive a string of characters from the server 
if (port.equals("443")){
while ((receiveMsg=httpsClient.receive())!=null){
//lambda You cannot directly access external expressions final Type local variables , You need to define a temporary variable
// If the receiveMsg Defined as a class member variable , You don't need a temporary variable 
String msgTemp = receiveMsg;
Platform.runLater(()->{
taDisplay.appendText(msgTemp+"\n");
});
}
}
});
readThread.start();
}
else if (port.equals("80")){
httpClient = new HTTPClient(ip, port);
taDisplay.appendText(" Server connection successful .\n");
readThread = new Thread(()-> {
String receiveMsg = null;
while ((receiveMsg = httpClient.receive()) != null) {
String msgTemp = receiveMsg;
Platform.runLater(() -> {
taDisplay.appendText(msgTemp + "\n");
});
}
});
readThread.start();
}
}catch (Exception e){
taDisplay.appendText(" Server connection failed !"+e.getMessage()+"\n");
}
});
// Web request button event 
btnSend.setOnAction(event -> {
String ip=ipAddress.getText().trim();
String port=tfport.getText().trim();
String domainName=ipAddress.getText().trim();
try {
StringBuffer msg = new StringBuffer();
msg.append("GET / HTTP/1.1\r\n"+
"HOST: "+domainName+"\r\n"+
"Accept: */*\r\n"+
"Accept-Language: zh-CN\r\n"+
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)\r\n"+
"Connection: Keep-Alive\r\n"
);
if (port.equals("443"))
httpsClient.send(msg.toString());
else if (port.equals("80"))
httpClient.send(msg.toString());
} catch (InterruptedException e) {
e.printStackTrace();
}
});
btnExit.setOnAction(event -> {
try {
exit();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
// Event in response to form closing , Click on the × close , The client is also shut down 
primaryStage.setOnCloseRequest(event -> {
try {
exit();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
}
private void exit() throws InterruptedException {
if (httpsClient!=null||httpClient!=null){
readThread.sleep(1000);// Multithread waiting , There are threads waiting to close the window IO, Set up 1s The interval ensures that all threads are shut down 
 httpsClient.close();
httpClient.close();
}
System.exit(0);
}
}
View Code

6、 ... and 、 Last + demonstration

HTTP Connect www.baidu.com, success

HTTP Connect www.sina.com.cn, Failure

HTTPS Connect www.sina.com.cn, success

Final review , By the way, write a blog , This is the first one , Introduce HTTP Web request download , Mainly HTTP(s) Network programming of protocol , Including the use of TCP Socket Three handshakes HTTP Website download , And use SSL Socket Safe transmission of HTTPs Website download , Complete the programming by case practice , know http(s) The actual working mechanism of !

expect :Java And HTTP Network programming ( The next part : Web browser programming ), Will see the page of HTML Source code , And the web page rendered by browser function !

 

If you think it's good, welcome to “ One key, three links ” Oh , Like, collect, pay attention to , Comment directly if you have questions , Communication and learning !


My blog Park :https://www.cnblogs.com/chenzhenhong/p/14435762.html

my CSDN Blog :https://blog.csdn.net/Charzous/article/details/111470556


 

Copyright notice : This article is an original blog article , follow  CC 4.0 BY-SA  Copyright agreement , For reprint, please attach the original source link and this statement .

 

Link to this article : https://blog.csdn.net/Charzous/article/details/111470556

 

版权声明
本文为[Charzueus]所创,转载请带上原文链接,感谢
https://qdmana.com/2021/02/20210223143351452C.html

  1. 对前端异常window error捕获的全面总结
  2. A comprehensive summary of front end exception window error capture
  3. 成功解决Problem while trying to mount target]\“. HTTP response code is 400
  4. Problem while trying to mount target] \ ". HTTP response code is 400
  5. 放弃okhttp、httpClient,选择了这个牛逼的神仙工具!贼爽
  6. 前端面试每日 3+1 —— 第679天
  7. How to add elements at the beginning of an array in JS?
  8. Give up okhttp and httpclient and choose this awesome immortal tool! Thief Shuang
  9. Front end interview daily 3 + 1 - day 679
  10. 【2021 第一期】日常开发 26 个常见的 JavaScript 代码优化方案
  11. Daily development of 26 common JavaScript code optimization schemes
  12. 前端的字符串时间如何自动转换为后端Java的Date属性,介绍springMVC中如何解决时间转换问题
  13. How to automatically convert the front-end string time to the back-end Java date attribute, and how to solve the time conversion problem in spring MVC are introduced
  14. 前端面试常考题:JS垃圾回收机制
  15. ReactDOM.render串联渲染链路(一)
  16. 更简洁、更快速!腾讯云 Serverless 云函数创建流程再次升级!
  17. 粗涉Webpack
  18. Frequently asked questions in front end interview: JS garbage collection mechanism
  19. ReactDOM.render Serial rendering link (1)
  20. More concise and faster! Tencent cloud serverless cloud function creation process upgrade again!
  21. 更简洁、更快速!腾讯云 Serverless 云函数创建流程再次升级!
  22. About webpack
  23. More concise and faster! Tencent cloud serverless cloud function creation process upgrade again!
  24. 详解vue静态资源打包中的坑与解决方案
  25. 一篇搞懂TCP、HTTP、Socket、Socket连接池
  26. 字节跳动2021前端技术岗发布+最新内部面试题
  27. Detailed explanation of Vue static resource packaging and Solutions
  28. Understanding TCP, HTTP, socket, socket connection pool
  29. 2008-2021 front end technical post release + latest internal interview questions
  30. 4. Vue基本指令
  31. 4. Vue basic instruction
  32. Java 发起 http 请求
  33. Java initiates HTTP request
  34. 网站由http升级为https图文教程
  35. Upgrade the website from HTTP to HTTPS
  36. 更简洁、更快速!腾讯云 Serverless 云函数创建流程再次升级!
  37. More concise and faster! Tencent cloud serverless cloud function creation process upgrade again!
  38. 混合开发入门 Vue结合Android/iOS开发仿京东项目App
  39. Hybrid development entry Vue combined with Android / IOS to develop app imitating Jingdong project
  40. 无缝对接 Tableau,这家月活跃用户 5000+ 的大型银行如何实现自助式分析?
  41. Are airpods still the strongest? Horizontal evaluation of 4 true wireless headphones
  42. Seamless connection with tableau, how can this large bank with 5000 + active users realize self-service analysis?
  43. react-native版文字跑马灯
  44. React native text running lantern
  45. Java、JavaScript、C、C++、PHP、Python都是用来开发什么?
  46. What are Java, JavaScript, C, C + +, PHP and python used to develop?
  47. this.byId(SupplierForm).bindElement in SAP UI5
  48. SAP UI5 JavaScript文件的lazy load - 懒加载
  49. this.byId (SupplierForm).bindElement in SAP UI5
  50. Lazy load lazy load of SAP ui5 JavaScript files
  51. "Gnome 3" - interface elements, desktop components, part names (learning notes) @ 20210223
  52. How to connect the ground gas to the micro front end?
  53. How to transform single / micro service application into serverless application
  54. 在 2021 年你需要掌握的 7 种关于 JavaScript 的数组方法
  55. Seven array methods for JavaScript you need to master in 2021
  56. 在 2021 年你需要掌握的 7 种关于 JavaScript 的数组方法
  57. Seven array methods for JavaScript you need to master in 2021
  58. 在 2021 年你需要掌握的 7 种关于 JavaScript 的数组方法
  59. Seven array methods for JavaScript you need to master in 2021
  60. RxHttp - 轻量级、可扩展、易使用、完美兼容MVVM、MVC架构的网络封装类库