Shukipel – Information ExchangeTech Information |
|
Glad to announce a brand new tools which will be hopefully helpful in analysing different networking and system issues.
First set of tools are DNS related. you can find the DNS Tools right here at http://www.shukipel.com/dns-tools.
In these set of tools you can query information such as reverse lookup, mx records, domain lookup, whois information, ns lookup, spf records verification and dkim verification.
Recently I had to convert a database of a system that used UTF-8 encoding but the database was default created with latin1.
All worked fine until greek letters was needed… than all greek characters were showing question marks (????)
main problem was that I already had data in the tables, so just recreating the database wouldnt cut it…
Research everywhere got me to a lot of different information but only one was close enough to help me solve my problem, so I decided to share this here with you.
This is under the assumption that you application connection to the database use UTF8 (set names)…
The article that had me start this was: http://en.gentoo-wiki.com/wiki/Convert_latin1_to_UTF-8_in_MySQL
The simple steps there were (BACKUP YOUR DATABASE BEFORE!!):
1. Dump the database:
mysqldump -p --default-character-set=latin1 -c \ --insert-ignore --skip-set-charset crm > dump.sql
2. Convert the dump file to utf8 using iconv
iconv -f ISO-8859-1 -t UTF-8 dump.sql > dump_utf8.sql
3. Change table definition from latin1 to utf8 using perl
perl -pi -w -e 's/CHARSET=latin1/CHARSET=utf8/g;' dump_utf8.sql
4. make sure you didnt forget anything
grep latin1 dump_utf8.sql
5. Create a new UTF8 Database
mysql --user=foo -p --execute="CREATE DATABASE NEWDB_utf8 CHARACTER SET utf8 COLLATE utf8_general_ci;"
6. Restore converted utf8 dump to the database
mysql --max_allowed_packet=16M -p --default-character-set=utf8 NEWDB_utf8< dump_utf8.sql
7. dont forget to grant privileges to application user
mysql --user=foo -p --execute="GRANT ALL PRIVILEGES on NEWDB_utf8.* to 'some_user'@'localhost' identified by 'some_pass';"
Now you are basically done. change connection configuration to the new DB name and see it working, HOWEVER you might see some characters were not encoded correctly and now showing as gibberish. here comes dirty work…
you need to check which character is screwed in your table and change it, for example:
character (’) is screwed up, you need to check where it appears in the tables and change it’s unicode value(C292):
To find:
select count(*) from some_table WHERE subject REGEXP UNHEX('C296');
To Replace with utf8 equivelent E28093:
UPDATE some_table SET subject= REPLACE(subject, UNHEX('C292'),UNHEX('E28099') ) WHERE subject REGEXP UNHEX('C292');
same routine for (–) C296 to E28093
hope you won’t find too many of thes at least now you have a solution without losing your data.
we all know or at least heard about asterisk by now. an open source PBX, which provide small and big enterprises the opportunity to cut costs and improve their old fashioned telephony systems.
the problem with old fashioned pbx systems made by avaya/alcatel/etc. is that they are way over priced, For years they had the ability to set the price for their systems license for almost every component in the system , and most of the times also force us to use their proprietary phones.
PBX based on Asterisk is first of all free, all you need to take care of is the hardware, ranging from a free vmware appliance on your office server for your small business to a dedicated heavy duty server for your call centre.
I recently implemented an asterisk based pbx for a 30 people office, inter connect it with 2 other locations with SIP trunks, and I can tell you… what a joy!
easy to configure, extensions were bulk loaded, PRI took 2 minutes to activate with the operator, 30 SNOM300 phones were configured easy and the office telco is up and running, free features such as: conference rooms,softphone video conversations, fax to email, and many more, no more licenses! upgrades are free, the only cost is per extension is a physical phone roughly $100 and if that person want a headset it’s even free using a softphone on the computer.
the pbx software is used there was elastix, and it’s a very very fun tool to use.
how much it costs?
software/licenses: FREE
Hardware: $2500 (for small office can be summed up by $1000 with PRI support, we chose to have it ready for expansion)
Phone:$100
softphone: FREE
all together (30 extensions): $5500
having this one implemented successfully, we decided to switch our main office from the old digital/analog pbx to the same system.
about 80 people with expectation to grow, the system cost us only double than what we should have paid to increase the capacity on our old pbx.
Elastix PBX Config
to sum it up, if you are afraid of switching your office telecommunications to an open source product, fear not! that system and i am only a small example is being implemented every day in avery complex places and scenarios.
the costs cut is unbelievable.
to sum it up GO asterisk GO!
FLASH Operator Panel
If you are an experienced “networking administrator”, UNIX/Linux Guru or a php/perl programmer and you are looking for a place to host your wisdom, we will be more than happy to give you a stage, for your applications ,tutorials or guides.
Please drop me an email to lior@shukipel.com
working closely with load balancers throughout the years made me aware of quite a few difficulties and tricks, and hard learned lessons about the problems and the best ways of implementing a load balancer into your web farm.
I hope you enjoy the things i have to say here, and i will listen to what you have to say , correct and update this article.
1. Web application, clock synchronization.
In load sharing setup, one must not count on the servers clocks to be used for writing data to the database or presenting data to the user.
The clock of every server skew, some servers skew more than others, so even a clock sync of once a day, doesn’t make it sure that the servers are synchronized, lets look at the example:
User login to your website, your application took the server’s time and inserted the login time as 14:00.
Now the user submits a form, and the second request is being sent through a different server, now another entry is being sent to the database at 13:59, because there was a few seconds difference between the clocks.
A later action was registered in the database as if it happened before the earlier one.
The best way to avoid it, is to use the database clock for all read and write actions, that ensures a steady time across various actions, through no matter how many servers you have in the pool.
2. application sessions type
If your application uses sessions, its better to use database based sessions, this will keep the user session alive no matter which server in your pool the request is being made from.
be sure to keep your servers equally loaded. you should be able to set your content group up as round robin with no stickiness.
If you must use server based session, then you must setup some kind of stickiness on your Load Balancer content group, so if you do, make sure the stickiness is based on source IP AND source port.
Pay attention, with some Load Balancers, when you chose sticky source IP they disregard the source port, this result in all requests coming from a certain IP to go to one server regardless of the amount of different requests, or different people initiating the requests, make sure the stickiness is based on source port as well, This will avoid search engines bots to load one of the servers while indexing your site, this could cause lack of performance on the loaded server.
4.server keep alive
if your Load Balancer support keep alive URI, which being used by the Load Balancer to make sure the server is alive, be sure to use a keep alive file which also uses the servers scripting engine(.php,.aspx,etc…) sometimes there is a problem with the scripting engine while the server itself is fine and still handing out responses, and you want that server to be removed from the pool and not resulting in application errors for your users.
Using a URI keep alive rather than other methods such as icmp, gives you the possibility to remove a server from the pool easily by renaming the keep alive file that the Load Balancer is checking for.
This can be used for maintenance, version updates, without doing any action on the Load Balancer itself.
5. Alerts
If possible, configure email/snmp traps alerts for major incidents, such as server unavailable, service unavailable,etc.
This helps identifying problems in the system, and to give you information when your environment status is being degraded.
6. Monitoring
Graph your interfaces, the bandwidth of each service, the hits of each service, and anything else you can possibly graph.
When investigating a fault, graphs of past behavior will help you to understand what could possibly went wrong.
7. Access lists
Setup access lists to restrict traffic going through the Load Balancer, maybe you are serving different internal networks, some Load Balancers by default behave like a router between different networks, which results in an unfiltered traffic between supposedly filtered networks.
Put explicitly allow access lists on all interfaces, and use logging, this will help identify access problems and/or unwanted traffic generated on the network.
Feel free to comment,
Lior.
