How do you search for downloaded data (a possible question from many users)

[barzifurze]

Hello

An important question which can come to your mind is: How do you search this large amount of data that we download?
A large amount of data is in csv format. and searching in this file takes a lot of time.

The tools that exist for this work:

The best option for this case is to use emeditor
which has many limitations for searching among hundreds of gigabytes of data
Cronos files are also available, but not for most databases, which have their own problems for searching

What solution do you use to search through this massive amount of downloaded data?
Do you have your own tool or is the tool available to the public?
Do you know a professional data search tool? If so, explain a little about it?

 

[Racoon]

I find emeditor not useful because it has so many limitations that I can really use it for the stuff I need

 

[barzifurze]

I find emeditor not useful because it has so many limitations that I can really use it for the stuff I need

Do you use a special tool?

 

[Racoon]

Do you use a special tool?

I usually use dataverse; you can look for it here:

Finding and Using Data — Dataverse.org

guides.dataverse.org

 

[Dread Pirate Roberts]

grep, sift

 

[laifu]

grep, sift

I also use those and recommend it

 

[faherya]

grep and awk. breached-parse is still good too: https://github.com/hmaverickadams/breach-parse

 

[Racoon]

grep and awk. breached-parse is still good too: https://github.com/hmaverickadams/breach-parse

Is breached-parse still works even after the seized?

 

[faherya]

Is breached-parse still works even after the seized?

This "breach parse" is a script, not the "breach parse" service. you can check the source code

 

[Racoon]

This "breach parse" is a script, not the "breach parse" service. you can check the source code

Got you bro thanks! I thought its something else, I will look into the script

 

[barzifurze]

Simple tool to set up a

Database search engine csv support

You know on the Windows platform

 

[th3tr0ll]

Strange no one seems to have suggested another great tool: glogg (https://glogg.bonnefon.org/)

I've been able to open / do searches on files bigger than 500+ GBs with it