The Open Guide to Glasgow scraping
https://wiki.glasgow.social/
2026-04-29T09:00:08+0100The Open Guide to Glasgow
https://wiki.glasgow.social/
https://wiki.glasgow.social/lib/tpl/dokuwiki/images/favicon.icotext/html2020-11-20T18:10:19+0100Anonymous (anonymous@undisclosed.example.com)scraping:dns_filter
https://wiki.glasgow.social/scraping/dns_filter?rev=1605895819&do=diff
Scraping: DNS Filter
This code takes the unique URLs (see Hidden Glasgow for an example and checks which ones are pointing to domains that no longer exists (then drops them from the to-be-checked list).
<?php
$file_list = "non-image-links.txt";
$data = file_get_contents($file_list);
$lines = explode("\n", $data);
$host_list = array();
foreach($lines as $url) {
$url_parts = parse_url($url);
if(!empty($url_parts['host'])) {
$host = $url_parts['host'];
…