The digital world is increasingly dominated by bots and not all of them are friendly. Many AI bots ignore robots.txt instructions, malicious crawlers overload servers, grab data and impair the performance of websites. At the same time, the risk of DDoS attacks increases and non-existent pages are crawled, leading to unnecessary server load.

For operators of content management systems (CMS), the question arises: How can you effectively protect your platform without penalising legitimate users? EGOCMS relies on an intelligent combination of bot blocking, caching strategies and automated protection mechanisms that are already integrated into the standard system.

The end of the robots.txt era

Since 1994, robots.txt has been considered a "courtesy rule" for crawlers. But modern AI bots ignore these guidelines to extract content on an industrial scale.

Crawling non-existent pages (404 floods)

Many bots scan not only existing but also non-existent URLs - whether through broken links, targeted attacks or automated scans. This leads to

Unnecessary server load due to repeated 404 requests
Performance losses for real visitors
Risk of DDoS-like conditions

DDoS attacks: When traffic becomes a weapon

Distributed Denial of Service (DDoS) attacks aim to paralyse websites with a flood of requests. Traditional defence mechanisms such as IP blocking or CAPTCHAs are often ineffective, as attackers constantly change their IPs or obstruct human users.

How EGOCMS protects by default

1. the EGOCMS bot brake: intelligent load control

EGOCMS relies on a dynamic bot brake that regulates bot access in real time:

Automatic throttling: bot requests are delayed or temporarily blocked when the server load is high.
Prioritisation of real users: Legitimate visitors and important services are always given priority.
No reliance on outdated protocols: Instead of relying on the ineffective robots.txt, EGOCMS analyses the current load and dynamically adjusts the access control.

"Every third website visitor is now a bot. The dynamic bot brake from EGOCMS ensures that your resources arrive where they are needed: with your real users." - EGOCMS Blog: Why bot management is indispensable today

2. protection from 404 floods

EGOCMS automatically recognises and blocks repeated 404 requests from the same IP:

After 5 failed attempts in 10 seconds, the IP is blocked for 10 seconds.
404 pages are cached to minimise the server load.
Smarty plugins respond with an HTTP 400 status if the requested page does not exist - this reduces unnecessary database queries.

3. nginx cache: performance boost and protection against overload

EGOCMS uses Nginx as a proxy cache to deliver static content at lightning speed and at the same time cushion attacks:

Only non-logged-in pages are cached in order to display personalised content correctly.
Automatic cache management: EGOCMS deletes the cache when changes are made and thus prevents outdated content.
HTTP header optimisation: The Cache-Control header is set dynamically to deactivate caching for logged-in users.

4. protection against DDoS through caching and rate limiting

Static content is delivered directly via Nginx without burdening the server with PHP or database accesses.
APCu cache for sections of websites that are repeated on different pages (navigation, header, footer) accelerates the delivery of dynamic content and reduces the database load.

Conclusion: Proactive protection as standard

EGOCMS shows how modern CMS design responds to current threats:

No dependence on outdated protocols such as robots.txt
Automated defence mechanisms against bots, 404 floods and DDoS
Performance optimisation through Nginx cache and APCu cache

For companies, universities and public institutions - the main target group of EGOCMS - this means

More stability through intelligent load balancing
Better performance for real users
Lower hosting costs thanks to efficient resource utilisation

Created by Roth Heiko yesterday at 09:00 o'clock