Bytespider is the web crawler used by Bytedance, the company behind platforms such as TikTok and Toutiao. Its main purpose is to browse and index publicly available web content, helping Bytedance understand website structure, content, and links. This information can then be used to improve search functionality, content recommendations, and other data-driven features across Bytedance’s services.
Although Bytespider is a legitimate crawler, it can generate noticeable traffic on websites, particularly large or frequently updated ones. Website owners can monitor its activity in server logs to ensure it functions as intended. Like other corporate web crawlers, Bytespider is generally safe, focusing on content analysis rather than malicious scraping or spamming.
ec2-47-128-35-43.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-57-145.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-42-29.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-31-178.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-99-119.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-116-174.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-35-15.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-60-98.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-18-135.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-119-45.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-48-86.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-33-194.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-56-189.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36
ec2-47-128-116-20.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-30-129.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-33-29.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-16-45.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-53-184.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
bytespider-111-225-148-135.crawl.bytedance.com
Expand
Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36
ec2-47-128-123-15.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-55-12.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36
ec2-47-128-50-176.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-21-188.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-17-125.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
ec2-47-128-23-42.ap-southeast-1.compute.amazonaws.com
Expand
Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36
