A piggy bank of commands, fixes, succinct reviews, some mini articles and technical opinions from a (mostly) Perl developer.

Jump to

Quick reference

Elasticsearch advanced queries

See also ElasticSearch basics.

DQL to filter by non-zero length: Advert.location_query:* (does not work as a filter)
Or in Lucene: Advert.location_query:?*

Results are limited to 10,000 records, unless you use the scroll API which can paginate and also make parallel requests.

Gist:
use strict;
use warnings;
use Data::Dumper::Concise;
use Search::Elasticsearch;
my $ekk = Search::Elasticsearch->new(
client => '7_0::Direct',
nodes => [ 'https://opensearch.example.com/', ],
send_get_body_as => 'POST',
$ENV{USE_EKK_PROXY} ? (handle_args => { https_proxy => $ENV{USE_EKK_PROXY} }) : (),
);
my $results = $ekk->search(
body => {
"size" => 500,
"sort" => [
{
"timestamp" => {
"order" => "desc",
"unmapped_type" => "boolean"
}
}
],
"aggs" => {
"2" => {
"date_histogram" => {
"field" => "timestamp",
"calendar_interval" => "1d",
"time_zone" => "Europe/London",
"min_doc_count" => 1
}
}
},
"stored_fields" => ["*"],
"script_fields" => {},
"docvalue_fields" => [
{
"field" => "timestamp",
"format" => "date_time"
}
],
"_source" => {
"excludes" => []
},
"query" => {
"bool" => {
"must" => [],
"filter" => [
{
"match_all" => {}
},
{
"match_phrase" => {
"foo_field" => "bar_value"
}
},
{
"range" => {
"timestamp" => {
"gte" => "2023-06-28T12:04:15.943Z",
"lte" => "2023-09-28T12:04:15.943Z",
"format" => "strict_date_optional_time"
}
}
}
],
"should" => [],
"must_not" => []
}
},
"highlight" => {
"pre_tags" => ["\@opensearch-dashboards-highlighted-field\@"],
"post_tags" => ["\@/opensearch-dashboards-highlighted-field\@"],
"fields" => {
"*" => {}
},
"fragment_size" => 2147483647
}
}
);
print Dumper($results);

MySQL date display format conversion

MySQL date time conversion functions: 

  • UNIX_TIMESTAMP(date) docs
  • FROM_UNIXTIME(epoch) docs

Examples:

select name, from_unixtime(time_added, '%Y-%m-%d %h:%i')

from company

order by date_added desc

limit 10; -- list the most recently added companies

You may also omit the '%Y-%m-%d %h:%i' format string, to get the default format YYYY-MM-DD HH:MM:SS, e.g. `2023-09-07 09:43:51`