Website Contacts Scraper

FREEMIUM
Verified
By OpenWeb Ninja | Updated 18 days ago | Data
Popularity

9.8 / 10

Latency

2,019ms

Service Level

100%

Health Check

100%

Back to All Discussions

1 Domain per keyword only

Rapid account: Ppkarnik
ppkarnik
10 months ago

For every keyword I enter, I get one domain only. How do I get more than 1?

Rapid account: Letscrape 6 B R Ba 3 Qgu O 5
letscrape-6bRBa3QguO5 Commented 10 months ago

Hi,

I am very glad you find this addition useful! Yes, less than 50 items can sometimes be returned if there are duplicates and / or there are less than 50 matches.

Let me know if anything else is needed.

Thank you,
OpenWeb Ninja

Rapid account: Ppkarnik
ppkarnik Commented 10 months ago

Hi OpenWeb Ninja,

I tested your new feature. Added “limit” as a parameter in the Request Body. Initially set it to “2” and it worked fine. Changed it to 50 to test the max value. Got a list / array for each keyword as you said. Interestingly, the list contained fewer than 50 items in each keyword. I am guessing that may be because internally you may have duplicates that are screened out. As it is, I am finding the functionality quite useful and usable.

Thanks,
Prasad

Rapid account: Ppkarnik
ppkarnik Commented 10 months ago

Thanks a lot for listening and adding useful functionality. I will test it out.

Best Regards,
Prasad

Rapid account: Letscrape 6 B R Ba 3 Qgu O 5
letscrape-6bRBa3QguO5 Commented 10 months ago

Hi,

  1. I added support for a new “limit” parameter (it doesn’t appear on the playground for now but you can use it), in case limit == 1, the response will be like today., otherwise, if limit > 1, the endpoint will return a list / array for each keyword (instead of a single string). The new limit parameter should be 0 < limit < 50 and is 1 by default. I hope this helps with what you need.

2 + 3. In case this is done on the API side, it’s going to filter the results before returning them to the caller, as if it was done on the client. Maybe an option could be to use the max limit of 50 and then do whatever is needed on the client with the complete list using a single API call, what do you think?

Thank you,
OpenWeb Ninja

Rapid account: Ppkarnik
ppkarnik Commented 10 months ago

#2 can be done on the client side but if the same domains are returned every time for a given keyword, then the filtering on the client side will render no new information. The thinking is that if the do-not-include input list is made available to the search process in your code, then it can ensure the results from the search are fresh.

#3 can also be done in the client side. Same issue as #2 when implemented on the client side.

Rapid account: Letscrape 6 B R Ba 3 Qgu O 5
letscrape-6bRBa3QguO5 Commented 10 months ago

Hi,

Thanks for your good suggestions!

I think this can be very helpful. Regarding #2 and#2, it sounds to me like those can be done client side (unlike #1), although I do understand it’ll be convenient. In any case, please let me look into this and get back to you.

Thank you,
OpenWeb Ninja

Rapid account: Ppkarnik
ppkarnik Commented 10 months ago

Yes, it would definitely help. A couple more suggestions:

  1. Add a limit on the number of results returned.
  2. Take as input a list of domains so that the results do not contain this list.
  3. Accept as input a regex pattern so that the domains listed match the list. You may end up returning no results too.
Rapid account: Letscrape 6 B R Ba 3 Qgu O 5
letscrape-6bRBa3QguO5 Commented 10 months ago

Hi,

Thank you for your question.

I assume you are referring to the Get Website by Keyword endpoint. It currently returns a single domain per keyword but I can introduce a flag to ask for multiple matches per keyword. Would that help with your use case?

Thank you,
OpenWeb Ninja

Join in the discussion - add comment below:

Login / Signup to post new comments