Avatar
# install java JRE (so solr can run)
[entadmin@dev3 equeuelib]$ sudo yum install java-1.8.0-openjdk
# enable it in riak.conf: search=on
# reload/restart riak as needed
#create index (can also be done via PHP)
[aesteban@localhost ansible]$ curl -X PUT riak.example.com:8098/search/index/story_index
# associate index to bucket (can also be done via PHP)
[aesteban@localhost ansible]$ curl -X PUT riak.example.com:8098/buckets/story/props -H'content-type:application/json' -d'{"props":{"search_index":"story_index"}}'
# verify it by viewing bucket props, search for 'search_index'
riak.example.com:8098/buckets/story/props
# add indexable fields to a Story object per se
eg: authorId_i and publicationDate_dt
# query it !
riak.example.com:8098/search/query/story_index?wt=json&q=authorId_i:8144
# it returns all story keys associated with that author.
# looks like there's sort capabilities available too:
riak.example.com:8098/search/query/story_index?wt=json&q=authorId_i:8144&sort=publicationDate_dt%20desc
# by default it seems to put a limit of 10 rows, we can change that...
riak.example.com:8098/search/query/story_index?wt=json&q=authorId_i:8144&sort=publicationDate_dt%20desc&rows=50
# Solr dashboard riak.example.com:8093/internal_solr/#/story_index
PHP
$indexes = [
[
"name" => "story_index",
"bucket" => "story"
]
];
$node = (new \Basho\Riak\Node\Builder)
->atHost('riak host')
->onPort('riak port'))
->build();
$riak = new \Basho\Riak ([$node]);
foreach ($indexes as $index)
{
// creates index
$response = (new \Basho\Riak\Command\Builder\Search\StoreIndex($riak))
->withName( $index['name'] )
->usingSchema('_yz_default')
->build()
->execute();
// associates index with bucket
(new \Basho\Riak\Command\Builder\Search\AssociateIndex($riak))
->withName( $index['name'] )
->buildBucket($index['bucket'])
->build()
->execute();
}
Docs:
http://docs.basho.com/riak/kv/2.2.3/dev ... ge/search/
https://docs.riak.com/riak/kv/latest/developing/usage/search/index.html
Default schema:
https://raw.githubusercontent.com/basho/yokozuna/develop/priv/default_schema.xml
12/7/2020
Sample query
$query = [
// "defType" => "edismax", // query parser
"sort" => "score DESC", // override default score sorting
"q" => "{!edismax}{$parameters['phrase']}", // query
"qf" => "authorName^6 objectId^4 headline^2 deck", // query fields
"fq" => "{!lucene}
edition:{$parameters['edition']}
AND statusId:4
AND objectTypeId:(1 2 4 12 15)
AND publicationDateISO8601:[NOW-10YEAR TO NOW]", // filter query
"qs" => "5", // query phrase slop
"bq" => "publicationDateISO8601:[NOW-2YEAR TO NOW]", // boost query
"fl" => "*,score", // field list
"hl" => "true", // highlight
"mm" => "3<80%", // minimum match
"wt" => "json", // response writer
"rows" => $rows, // rows
"start" => $start, // offset
// spell check parameters
"df" => "entspellcheck",
"spellcheck" => "true" ,
"spellcheck.q" => "\"{$parameters['phrase']}\"~10" , // proximity added
// "spellcheck.build" => "true" ,
"spellcheck.collate" => "true" ,
"spellcheck.maxCollations" => "30" ,
"spellcheck.maxCollationTries" => "30" ,
"spellcheck.maxCollationEvaluations" => "30" ,
"spellcheck.collateExtendedResults" => "true" ,
"spellcheck.collateMaxCollectDocs" => "30" ,
"spellcheck.count" => "10" ,
// "spellcheck.dictionary" => 'wordbreak' ,
"spellcheck.extendedResults" => "true" ,
// "spellcheck.onlyMorePopular" => "true" ,
"spellcheck.maxResultsForSuggest" => "5" ,
"spellcheck.alternativeTermCount" => "10" ,
// "spellcheck.reload" => "true",
"spellcheck.accuracy" => "0.5" ,
];
Which will be something like this:
http://app01.example.com:8098/search/query/article_index?sort=score DESC
&q={!edismax}how to become a millionaire
&qf=authorName^6 objectId^4 headline^2 deck
&fq={!lucene}
edition:us
AND statusId:4
AND objectTypeId:(1 2 4 12 15)
AND publicationDateISO8601:[NOW-10YEAR TO NOW]
&qs=5
&bq=publicationDateISO8601:[NOW-2YEAR TO NOW]
&fl=*,score
&hl=true
&mm=3<80%
&wt=json
&rows=20
&start=0
&df=entspellcheck
&spellcheck=true
&spellcheck.q="how to become a millionaire"~10
&spellcheck.collate=true
&spellcheck.maxCollations=30
&spellcheck.maxCollationTries=30
&spellcheck.maxCollationEvaluations=30
&spellcheck.collateExtendedResults=true
&spellcheck.collateMaxCollectDocs=30
&spellcheck.count=10
&spellcheck.extendedResults=true
&spellcheck.maxResultsForSuggest=5
&spellcheck.alternativeTermCount=10
&spellcheck.accuracy=0.5
[ view entry ] ( 673 views ) | print article
This is a working draft, like lots of things in this site. For dev purposes firewalld was turned off.
Cluster VIP:172.16.1.213
TODO: VRRP failover setup for load balancer (keepalived)
ALSO: It looks like I configured net.ipv4.ip_forward=1 in load balancer but failed to document it, check this when doing a new load balancer.
#####################Edge01 Configuration (loadbalancer 1)#########################
//keepalived.conf
! Configuration File for keepalived
global_defs {
# notification_email {
# me[at]example.net
# }
# notification_email_from not-replay[at]example.net
# smtp_server 192.168.200.1
# smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
# 172.16.1.213
# 172.16.1.254
}
}
virtual_server 172.16.1.213 8098 {
delay_loop 6
lb_algo wlc
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.16.1.113 8098 {
weight 1
TCP_CHECK {
connect_timeout 3
connect_port 8098
}
}
real_server 172.16.1.112 8098 {
weight 100
TCP_CHECK {
connect_timeout 3
connect_port 8098
}
}
real_server 172.16.1.111 8098 {
weight 100
TCP_CHECK {
connect_timeout 3
connect_port 8098
}
}
}
//ifconfig (secondary address for VIP)
[acool@edge01 ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:24:54:3A:66:4D
inet addr:172.16.1.200 Bcast:172.16.1.255 Mask:255.255.255.0
inet6 addr: fe80::224:54ff:fe3a:664d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:385 errors:0 dropped:0 overruns:0 frame:0
TX packets:243 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:37042 (36.1 KiB) TX bytes:35302 (34.4 KiB)
eth0:0 Link encap:Ethernet HWaddr 00:24:54:3A:66:4D
inet addr:172.16.1.213 Bcast:172.16.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b)
[acool@edge01 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:24:54:3a:66:4d brd ff:ff:ff:ff:ff:ff
inet 172.16.1.200/24 brd 172.16.1.255 scope global eth0
inet 172.16.1.213/24 brd 172.16.1.255 scope global secondary eth0:0
inet6 fe80::224:54ff:fe3a:664d/64 scope link
valid_lft forever preferred_lft forever
#####################Real Server 2 ( 99% same output for other real servers) #########################
//DR-IPVS file
#!/bin/bash
VIP=172.16.1.213
host=`/bin/hostname`
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
/sbin/ifconfig lo:1 $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:1
exit 1
//ifconfig
[acool@app02 ~]$ ifconfig
ens1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.1.112 netmask 255.255.255.0 broadcast 172.16.1.255
inet6 fe80::216:d4ff:fe45:8ae1 prefixlen 64 scopeid 0x20<link>
ether 00:16:d4:45:8a:e1 txqueuelen 1000 (Ethernet)
RX packets 564 bytes 53931 (52.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 1
TX packets 428 bytes 57911 (56.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 13 bytes 1360 (1.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13 bytes 1360 (1.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo:1: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 172.16.1.213 netmask 255.255.255.255
loop txqueuelen 0 (Local Loopback)
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 172.16.1.213/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:16:d4:45:8a:e1 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.112/24 brd 172.16.1.255 scope global ens1
valid_lft forever preferred_lft forever
inet6 fe80::216:d4ff:fe45:8ae1/64 scope link
valid_lft forever preferred_lft forever
edge01 verification (keealived load balancer):
[acool@edge01 ~]$
[acool@edge01 ~]$
[acool@edge01 ~]$ sudo ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.1.213:8098 wlc persistent 50
-> 172.16.1.111:8098 Route 100 1 0
-> 172.16.1.112:8098 Route 100 0 0
-> 172.16.1.113:8098 Route 1 0 0
[acool@edge01 ~]$
[acool@edge01 ~]$
[acool@edge01 ~]$
Riak ring verification:
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$ sudo riak-admin member_status
================================= Membership ==================================
Status Ring Pending Node
-------------------------------------------------------------------------------
valid 32.8% -- 'riak@172.16.1.111'
valid 32.8% -- 'riak@172.16.1.112'
valid 34.4% -- 'riak@172.16.1.113'
-------------------------------------------------------------------------------
Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$ sudo riak-admin ring-status
================================== Claimant ===================================
Claimant: 'riak@172.16.1.113'
Status: up
Ring Ready: true
============================== Ownership Handoff ==============================
No pending changes.
============================== Unreachable Nodes ==============================
All nodes are up and reachable
[acool@app02 ~]$
Finally, storing data in riak:
Angels-Mac-mini:~ webmaster$ curl -v -XPUT 172.16.1.213:8098/buckets/test-bucket/keys/friends?returnbody=true -H "Content-Type: application/json" -d '{"Name":"Angel Cool"}'Fetching data:
http://172.16.1.213:8098/buckets/test-bucket/keys/friends
Or using any of the real servers:
http://172.16.1.111:8098/buckets/test-bucket/keys/friends
Adding another node at a later time:
[webmaster@dev ~]$ sudo riak-admin cluster join riak@172.16.1.111
[webmaster@dev ~]$ sudo riak-admin cluster plan
[webmaster@dev ~]$ sudo riak-admin cluster commit
If plan does not look good:
[webmaster@dev ~]$ sudo riak-admin cluster clear
Riak must be bound to all interfaces: 0.0.0.0:8098
[aesteban@localhost ~]$
[aesteban@localhost ~]$ riak-admin status | grep riak_kv_version
riak_kv_version : <<"1.4.7-0-g5daeedd">>
[aesteban@localhost ~]$
[ view entry ] ( 1685 views ) | print article
//list all buckets
curl -i riak1.barney.com:8098/buckets?buckets=true
//putting/storing JSON in riak
curl -v -XPUT riak1.barney.com:8098/buckets/my_bucket/keys/my_test?returnbody=true \
-H "Content-Type: application/json" \
-d '{"TestName":"Monday test"}'
//listing keys in bucket
curl -i riak1.barney.com:8098/buckets/my_bucket/keys?keys=true
//fetch a key
curl -v riak1.barney.com:8098/buckets/my_bucket/keys/my_test
http://docs.basho.com/riak/latest/dev/using/basics/
[ view entry ] ( 1356 views ) | print article
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |