ldb-api

Url’s Extractor and Details Scrapper

Meta, Open Graph, html tags, SEO data, urls contained, duration if YT video

Endpoints:

POST /json

file: example.txt

Post .txt file and get list of it’s urls

GET /json

?urls=https://example.com%20https://second-example.com

Get urls details from urls query

Examples of response:

{
    "urls": [
        {
            "url": "https://ogp.me",
            "title": "The Open Graph protocol",
            "desc": "The Open Graph protocol enables any web page to become a rich object in a social graph.",
            "ogImg": {
                "ogImg": "https://ogp.me/logo.png",
                "ogImgAlt": "The Open Graph logo"
            }
        },
        {
            "url": "https://developer.mozilla.org/",
            "title": "MDN Web Docs",
            "desc": "The MDN Web Docs site provides information about Open Web technologies including HTML, CSS, and APIs for both Web sites and progressive web apps.",
            "ogImg": {
                "ogImg": "https://developer.mozilla.org/mdn-social-share.cd6c4a5a.png",
                "ogImgAlt": ""
            }
        }
    ]
}

&advanced=true

{
    "urls": [
        {
            "url": "https://ogp.me",
            "title": "The Open Graph protocol",
            "desc": "The Open Graph protocol enables any web page to become a rich object in a social graph.",
            "keywords": "",
            "ogImg": {
                "ogImg": "https://ogp.me/logo.png",
                "ogImgAlt": "The Open Graph logo"
            },
            "ogDesc": "The Open Graph protocol enables any web page to become a rich object in a social graph.",
            "ogTitle": "Open Graph protocol",
            "canonical": "",
            "robots": "",
            "yTDur": "",
            "urls": [
                "https://ogp.me/",
                "https://microformats.org/"
            ],
            "htmlTags": {
                "h1": [
                    "<h1>The Open Graph protocol</h1>"
                ],
                "h2": [
                    "<h2><a href>Introduction</a></h2>",
                    "<h2><a href>Basic Metadata</a></h2>",
                   ...
                ],
                "h3": [
                    "<h3><a href>Optional Metadata</a></h3>",
                    "<h3><a href>Music</a></h3>",
                   ...
                ],
                "h4": [],
                "h5": [],
                "h6": [],
                "p": [
                    "<p>The <a href>Open Graph protocol</a> enables any web page to become a rich object in a social graph. For instance, this is used on Facebook to allow any web page to have the same functionality as any other object on Facebook.</p>",
                    "<p>The following properties are optional for any object and are generally recommended:</p>",
                    "<p>For example (line-break solely for display purposes):</p>",
                    ...
                ],
                "a": [
                    "<a href>Introduction</a>",
                    "<a href>Open Graph protocol</a>",
                    ...
                ],
                "img": []
            }
        },
        {
            "url": "https://developer.mozilla.org/",
            "title": "MDN Web Docs",
            "desc": "The MDN Web Docs site provides information about Open Web technologies including HTML, CSS, and APIs for both Web sites and progressive web apps.",
            "keywords": "",
            "ogImg": {
                "ogImg": "https://developer.mozilla.org/mdn-social-share.cd6c4a5a.png",
                "ogImgAlt": ""
            },
            "ogDesc": "The MDN Web Docs site provides information about Open Web technologies including HTML, CSS, and APIs for both Web sites and progressive web apps.",
            "ogTitle": "MDN Web Docs",
            "canonical": "https://developer.mozilla.org",
            "robots": "index, follow",
            "yTDur": "",
            "urls": [
                "https://twitter.com/mozdevnet",
                "https://discord.gg/hkGN8VKvvD",
                "https://hacks.mozilla.org/",
                "https://www.mozilla.org/",
                "https://www.mozilla.org",
                "https://foundation.mozilla.org/"
            ],
            "htmlTags": {
                "h1": [
                    "<h1>Resources for <u>Developers</u>,<br /> by Developers</h1>"
                ],
                "h2": [
                    "<h2>Featured Articles</h2>",
                    "<h2>Latest news</h2>",
                   ...
                ],
                "h3": [
                    "<h3><a href>New reference pages on MDN for JavaScript regular expressions</a></h3>",
                    "<h3><a href>Using HTML landmark roles to improve accessibility</a></h3>",
                   ...
                ],
                "h4": [],
                "h5": [],
                "h6": [],
                "p": [

                    "<p>The font-palette CSS property allows specifying one of the many palettes contained in a font that a user agent should use for the font. Users can also override the values in a palette or create a new palette by using the @font-palette-values at-rule.</p>",
                   ...
                ],
                "a": [

                    "<a href>mdn/content</a>",
                    "<a href>fix typos</a>",
                    "<a href>mdn/content</a>",
                   ...
                ],
                "img": []
            }
        }
    ]
}