morrigan/crabbysearch
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

✨ feat(config): config option to keep tcp connection alive for a certain period for subsequent requests (#548)

Browse source 
* Added new HTTP connection setting to the reqwest::ClientBuilder to timeout requests for fetching the search results from the upstream search engines.

* Provided a config option under the server section of the config to allow users to keep tcp connections alive for each request for a certain period of time

* Update src/results/aggregator.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Update src/results/aggregator.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Fixed import issue in `tcp_connection_keepalive`

* updated size to u8

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Fixed sizer eror in `parser.rs`

---------

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
Co-authored-by: alamin655 <129589283+alamin655@users.noreply.github.com>
This commit is contained in:
Kekma 2024-03-25 14:41:56 +05:30 • committed by GitHub
parent ce4912b9c5
commit c584a7d601
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
src/results/aggregator.rs
View file

@ -77,6 +77,7 @@ pub async fn aggregate(
let client = CLIENT.get_or_init(|| {
ClientBuilder::new()
.timeout(Duration::from_secs(config.request_timeout as u64)) // Add timeout to request to avoid DDOSing the server
.tcp_keepalive(Duration::from_secs(config.tcp_connection_keepalive as u64))
.connect_timeout(Duration::from_secs(config.request_timeout as u64)) // Add timeout to request to avoid DDOSing the server
.https_only(true)
.gzip(true)