Projet

Général

Profil

Scripts et taches planifiees » Historique » Version 73

Julien Enselme, 06/08/2013 13:12
La sauvegarde des bases de données : correction d'une phrase

1 1 Julien Enselme
Afin de gagner du temps et d'éviter les erreurs humaines, des scripts ont été écrits tout au long du projet. Certains sont même exécutés automatiquement toutes les semaines.
2
3 13 Florent Torregrosa
{{>toc}}
4 1 Julien Enselme
5
h1. Les tâches planifiées
6
7 26 Florent Torregrosa
h2. Crontab
8
9 3 Julien Enselme
Pour exécuter ces tâches régulièrement, on utilise le "crontab":http://fr.wikipedia.org/wiki/Crontab. Il s'agit d'un programme installé sur notre serveur.
10 1 Julien Enselme
11 3 Julien Enselme
Pour voir et modifier la liste des actions :
12
# se connecter au serveur : <code>ssh assos@sas1.centrale-marseille</code>
13
# taper la commande pour voir le crontab <code>crontab -l</code>
14 47 Julien Enselme
# taper la commande pour modifier le crontab <code>crontab -e</code>.
15 1 Julien Enselme
16 47 Julien Enselme
{{important(Ne pas effectuer cette opération avant de s'être renseigné sur "vi":http://fr.wikipedia.org/wiki/Vi (l'éditeur de texte utilisé qui n'est pas vraiment intuitif :p) et sur "la syntaxe du crontab":http://fr.wikipedia.org/wiki/Crontab#Modification !
17
Les scripts doivent si possible s’exécuter avant le club Drupal pour que les problèmes éventuels y soient réglés)}}
18
19 69 Julien Enselme
{{tip(
20
* Pour utiliser un autre éditeur que vi taper : export EDITOR=editor_name exemple : vim, nano.
21
* d7-all-update-contrib.sh fait des sauvegardes automatiques individuelles pour les sites drupal)}}
22 26 Florent Torregrosa
23
<pre>
24
<code class="php">
25 42 Florent Torregrosa
######## How to use
26
# mm hh jj MMM JJJ task
27
# minutes hour day_month month day_week
28
# for more informations : https://fr.wikipedia.org/wiki/Crontab#Notation
29
######## End how to
30
31 1 Julien Enselme
32
### Cleaning
33 67 Julien Enselme
00 5 * * * /users/guest/assos/bin/all-reset-permissions.sh > /dev/null 2>&1
34
00 6 * * * /users/guest/assos/bin/delete-temporary-files.sh > /dev/null 2>&1
35 1 Julien Enselme
36
37 67 Julien Enselme
### Traqu-heure
38
00 7 * * * /users/guest/assos/bin/traqu_heure/script-restitution.sh > /dev/null 2>&1
39 68 Florent Torregrosa
0 * * * *  /users/guest/assos/bin/traqu_heure/script-traque.sh > /dev/null 2>&1
40 67 Julien Enselme
41
42 1 Julien Enselme
### Databases
43
# Create backup
44 67 Julien Enselme
00 8 1,15 * * /users/guest/assos/bin/d7-all-dump-full.sh > /dev/null 2>&1
45 68 Florent Torregrosa
00 1 * * 2    /users/guest/assos/bin/d6-all-dump-full.sh > /dev/null 2>&1
46 43 Florent Torregrosa
47
# flush backup
48 67 Julien Enselme
00 8 2,16 * * /users/guest/assos/bin/flush-full-databases-backups.sh d7 > /dev/null 2>&1
49 68 Florent Torregrosa
00 22 * * 2   /users/guest/assos/bin/flush-full-databases-backups.sh d6 > /dev/null 2>&1
50
00 23 * * 2   /users/guest/assos/bin/flush-individual-databases-backups.sh auto d7 > /dev/null 2>&1
51
00 1 * * 3    /users/guest/assos/bin/flush-individual-databases-backups.sh manual d7 > /dev/null 2>&1
52 42 Florent Torregrosa
53 43 Florent Torregrosa
54 67 Julien Enselme
### Drupal 7
55 56 Julien Enselme
00 22 * * 3 /users/guest/assos/bin/d7-all-update-contrib.sh | mail -s "d7-all-update-contrib.sh" assos
56 68 Florent Torregrosa
00 1 * * 4  /users/guest/assos/bin/d7-all-update-localisation.sh | mail -s "d7-all-update-localisation.sh" assos
57
00 2 * * 4  /users/guest/assos/bin/d7-all-security-review.sh
58
00 3 * * 4  /users/guest/assos/bin/d7-all-reset-variables.sh > /dev/null 2>&1
59
00 4 * * 4  /users/guest/assos/bin/d7-all-releases-info.sh | mail -s "d7-all-releases-info.sh" assos
60 42 Florent Torregrosa
61 67 Julien Enselme
62 43 Florent Torregrosa
### Drupal 6
63 67 Julien Enselme
00 22 * * 6 /users/guest/assos/bin/d6-all-update-contrib.sh > /dev/null 2>&1
64 42 Florent Torregrosa
00 23 * * 6 /users/guest/assos/bin/d6-all-reset-variables.sh > /dev/null 2>&1
65 43 Florent Torregrosa
66 67 Julien Enselme
# Launch drush cron on fellowings
67 68 Florent Torregrosa
45 17 * * *  /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.ginfo > /dev/null 2>&1
68
*/15 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.clubfinance > /dev/null 2>&1
69
45 3 * * *   /users/guest/assos/bin/drushcronone default > /dev/null 2>&1
70
35 00 * * *  /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.cac13 > /dev/null 2>&1
71
*/45 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.expat > /dev/null 2>&1
72
*/10 * * * * /users/guest/assos/bin/drushcronone forum.centrale-marseille.fr > /dev/null 2>&1
73 67 Julien Enselme
74 1 Julien Enselme
00 22 * * 7 /users/guest/assos/bin/disk-usage.sh | mail -s "disk-usage.sh" assos 
75
</code>
76 25 Florent Torregrosa
</pre>
77 1 Julien Enselme
78 67 Julien Enselme
La liste des tâches effectuées régulièrement est ci-dessous.
79 3 Julien Enselme
80 25 Florent Torregrosa
h2. La réinitialisation des variables
81
82 1 Julien Enselme
Cette tâche consiste à réinitialiser certaines variables qui donnent des droits considérés comme trop permissifs donc dangereux aux administrateurs des sites ou nécessaire pour d'autres scripts tels que piwik ou security_review.
83
84 25 Florent Torregrosa
En voici quelques exemples :
85 1 Julien Enselme
86
<pre>
87
<code class="bash">
88 25 Florent Torregrosa
drush -y vset error_level 0
89
</code>
90 1 Julien Enselme
</pre>
91 21 Florent Torregrosa
92 25 Florent Torregrosa
Cette commande permet de ne pas afficher les messages d'erreurs. En effet, ils contiennent parfois des informations sensibles sur l'installation et ne doivent donc pas être divulgués à n'importe qui.
93
94 3 Julien Enselme
<pre>
95 1 Julien Enselme
<code class="php">
96
drush -y vset --always-set reverse_proxy TRUE
97 25 Florent Torregrosa
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
98 1 Julien Enselme
</code>
99 3 Julien Enselme
</pre>
100 1 Julien Enselme
101 70 Julien Enselme
Ces commandes permettent de déclarer à Drupal les serveurs proxy du CRI afin d'éviter qu'il ne répertorie tous les visiteurs comme ayant l'adresse des serveurs sus-cités. **Pas fini : voir http://assos.centrale-marseille.fr/lessive/content/t%C3%A2che/rendre-les-settingsphp-conformes-%C3%A0-la-version-712-de-drupal**
102 1 Julien Enselme
103 6 Julien Enselme
h2. La sauvegarde des bases de données
104 1 Julien Enselme
105 3 Julien Enselme
Sur les installations d6 et d7 : une fois par semaine
106 1 Julien Enselme
107 73 Julien Enselme
On utilise les scripts de sauvegardes :
108
109
* [[Scripts_et_taches_planifiees#d6-all-dump-fullsh|d6-all-dump-full.sh]]
110
* [[Scripts_et_taches_planifiees#d6-dump-individualsh|d6-dump-individual.sh]]
111
* [[Scripts_et_taches_planifiees#d7-all-dump-fullsh|d7-all-dump-full.sh]]
112
* [[Scripts_et_taches_planifiees#d7-all-dump-individualsh|d7-all-dump-individual.sh]].
113 1 Julien Enselme
114 72 Julien Enselme
h2. La purge des sauvegardes de bases de données
115 1 Julien Enselme
116
Sur les installations d6 et d7 : toutes les semaines
117
118 72 Julien Enselme
Cette tâche utilise les scripts de purges des sauvegardes ([[Scripts_et_taches_planifiees#flush-full-databases-backupssh|flush-full-databases-backups.sh]] et [[Scripts_et_taches_planifiees#flush-individual-databases-backupssh|flush-individual-databases-backups.sh]]) afin de libérer de l'espace disque en supprimant les sauvegardes de bdd les plus vieilles.
119 1 Julien Enselme
120
h1. Liste des scripts à disposition
121 21 Florent Torregrosa
122
Les scripts utilisés sont hébergés dans le répertoire bin du compte assos.
123
124 53 Julien Enselme
h2. Configuration et utilitaires
125 1 Julien Enselme
126 53 Julien Enselme
h3. scripts-config.sh
127
128
<pre>
129
<code class="php">
130
#!/bin/sh
131
132
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/users/guest/assos/bin
133
PATH=$PATH:/usr/guest/assos/bin
134
135
scripts_config='imported'
136
137
email_multi_assos="assos@centrale-marseille.fr"
138
139
dir_multi_assos='/users/guest/assos'
140
dir_scripts="$dir_multi_assos/bin"
141
dir_template="$dir_multi_assos/template"
142
143
###### Database
144
db_server="myassos.serv.int"
145
db_user="assos"
146
dir_log="$dir_multi_assos/log"
147
148
###### Backup
149
db_full_backup_number=3
150 63 Julien Enselme
db_individual_manual_backup_number=2
151 53 Julien Enselme
db_individual_auto_backup_number=3
152
153
dir_backup="$dir_multi_assos/backup"
154
dir_full_backup="$dir_backup/full"
155
dir_individual_backup="$dir_backup/individual"
156 1 Julien Enselme
dir_individual_manual_backup="$dir_individual_backup/manual"
157
dir_individual_auto_backup="$dir_individual_backup/auto"
158
159
160
###### D7 variables
161
d7_dir="$dir_multi_assos/htmltest"
162
d7_dir_sites="$d7_dir/sites"
163
164
### Log
165
d7_dir_log="$dir_log/d7"
166
167
### Backup
168
d7_dir_full_backup="$dir_full_backup/d7"
169
d7_dir_individual_manual_backup="$dir_individual_manual_backup/d7"
170
d7_dir_individual_auto_backup="$dir_individual_auto_backup/d7"
171
172
### Template
173
d7_settings_template_name="settings-D7-bddinde-template.php"
174
d7_settings_template="$dir_template/$d7_settings_template_name"
175
176
177
###### D6 variables
178
d6_dir="$dir_multi_assos/html"
179
d6_dir_sites="$d6_dir/sites"
180
181
### Backup
182
d6_dir_full_backup="$dir_full_backup/d6"
183
d6_dir_individual_manual_backup="$dir_individual_manual_backup/d6"
184
d6_dir_individual_auto_backup="$dir_individual_auto_backup/d6"
185 64 Julien Enselme
</code>
186
</pre>
187
188
h3. scripts-config-site.sh
189
190
<pre>
191
<code class="php">
192
#!/bin/sh
193
194
# Sets useful variables of a site.
195
# ARGS: site_name
196
# Requires scripts-config.sh
197
198
# Check if scripts-config.sh is imported.
199
if [ -z $scripts_config ] ; then
200
    echo "Import of scripts-config.sh required."
201
    exit 1
202
fi
203
204
if [ -z $1 ] ; then
205
    echo "This script needs a site name as argument."
206
    exit 1
207
fi
208
209
scripts_config_site='imported'
210
211
d7_site_name=$1
212
d7_site_dir="$d7_dir_sites/assos.centrale-marseille.fr.$d7_site_name"
213
d7_site_settings="$d7_site_dir/settings.php"
214 53 Julien Enselme
</code>
215
</pre>
216
217
h3. scripts-utils.sh
218
219
<pre>
220
<code class="php">
221
#!/bin/sh
222
223
# This script contains useful functions for other scripts.
224
225
# Check if scripts-config.sh is imported.
226
if [ -z $scripts_config ] ; then
227
    echo "Import of scripts-config.sh required."
228 58 Julien Enselme
    . scripts-config.sh
229 53 Julien Enselme
fi
230
231
scripts_utils='imported'
232
233
ask_password() {
234
    # read -s doesn't work with sh.
235
    # usage: pass=`ask_password "password please:"`
236
    echo $1 >&2
237
    echo -n ">" >&2
238
    stty_avant=`stty -g`
239
    stty -echo
240
    read password
241
    stty $stty_avant
242
    echo "$password"
243
    unset password
244
}
245
246
ask_password_db() {
247
    # ARGS: server_name, user_name
248
    local db_password="pour_boucler"
249
    # empty db request to validate password
250
    while ! mysql -h $1 -u $2 -p$db_password -e "" 2>/dev/null ; do
251
        db_password=`ask_password "database password:"`
252
    done
253
    echo $db_password
254
}
255
256
generate_password(){
257
    # ARGS: [password_length]
258
    # The password contains special characters. '/' must be excluded to avoid sed malfunction.
259
260
    local site_password='/'
261
262
    if [ -z $1 ] ; then
263
        local password_length=20
264
    else
265
        local password_length=$1
266
    fi
267
268
    while echo "$site_password" | grep -Fq '/' ; do
269
        site_password=`dd if=/dev/urandom count=1 | uuencode -m - | head -n 2 | tail -n 1 | cut -c-$password_length`
270
    done
271
272
    echo $site_password
273
}
274
275
count_d7_sites(){
276
    find $d7_dir_sites -type d ! -name all -maxdepth 1 | wc -l
277
}
278
279
check_arguments() {
280 65 Julien Enselme
    # ARGS: number of arguments passed to script, number of arguments required, [help text]
281 53 Julien Enselme
    if [ $1 -lt $2  ] ; then
282 1 Julien Enselme
        echo "Number of arguments insuffisant."
283 65 Julien Enselme
	echo $3
284 53 Julien Enselme
        exit 1
285
    fi
286
}
287
</code>
288
</pre>
289
290
h2. Scripts généraux
291
292
h3. all-reset-permissions.sh
293
294
<pre>
295
<code class="php">
296
#!/bin/sh
297
298 21 Florent Torregrosa
. scripts-config.sh
299
300
# This script puts the correct permissions to sites folders, settings.php and scripts.
301
302
######### drupal 6
303
for dir in `find $d6_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all ! -name images ! -name languages` ; do
304
    chmod 755 $dir
305
    chmod 400 $dir/settings.php
306 35 Florent Torregrosa
done
307
308
######### drupal 7
309
for dir in `find $d7_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all` ; do
310
    chmod 755 $dir
311
    chmod 400 $dir/settings.php
312 21 Florent Torregrosa
done
313
314 1 Julien Enselme
####### bin
315 21 Florent Torregrosa
chmod -R 700 $dir_scripts
316
317
####### backup
318
chmod -R 700 $dir_backup
319
320
####### log
321
chmod -R 700 $dir_log
322
</code>
323 1 Julien Enselme
</pre>
324 21 Florent Torregrosa
325 53 Julien Enselme
h3. contrib-usage.sh
326 29 Florent Torregrosa
327 21 Florent Torregrosa
<pre>
328
<code class="php">
329
#!/bin/sh
330
331
. scripts-config.sh
332
. scripts-utils.sh
333
334 66 Julien Enselme
help="# ARGS: Drupal_version, project_status, project_name\n
335
# List sites that have the project_name with the corresponding project_status.\n
336
# project status: enabled or disabled"
337 21 Florent Torregrosa
338 66 Julien Enselme
check_arguments $# 3 "$help"
339 21 Florent Torregrosa
340
if [ $1 = d6 ] ; then
341
    cd $d6_dir_sites
342
else
343
    cd $d7_dir_sites
344
fi
345 29 Florent Torregrosa
346 21 Florent Torregrosa
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort ` ; do
347
    # List non-core projects that correspond to the status.
348
    # Keep project_name if listed.
349
    # Count line result. 0 if not listed or 1 if listed.
350 1 Julien Enselme
	# Print site_dir if listed.
351 21 Florent Torregrosa
352
    cd $dir;
353
	if [ 1 -eq `drush pml --no-core --status=$2 | grep $3 | wc -l` ] ; then
354
        echo $dir; 
355
    fi
356
    cd -
357
done
358 1 Julien Enselme
</code>
359 21 Florent Torregrosa
</pre>
360
361 53 Julien Enselme
h3. delete-temporary-files.sh
362 21 Florent Torregrosa
363 41 Florent Torregrosa
<pre>
364 21 Florent Torregrosa
<code class="php">
365 1 Julien Enselme
#!/bin/sh
366
367
. scripts-config.sh
368 53 Julien Enselme
369
find $dir_multi_assos -name '*~' -delete
370
</code>
371
</pre>
372
373
h3. disk-usage.sh
374
375
<pre>
376
<code class="php">
377
#!/bin/sh
378
379
. scripts-config.sh
380
381
echo -e "Quotas: 9.1 GB\n"
382
383
echo "Assos:"
384
du -hcs $dir_multi_assos | grep total
385
echo -e "\n"
386
387
echo "D6:"
388
cd $d6_dir_sites
389
du -h -d 1
390
echo -e "\n"
391
392
echo "D7:"
393
cd $d7_dir_sites
394
du -h -d 1
395
echo -e "\n"
396
397
echo "Logs:"
398
du -hcs $dir_log | grep total
399
echo -e "\n"
400
401
echo "Backups:"
402
cd $dir_backup
403
du -h -d 2
404
</code>
405
</pre>
406
407
h3. drushcronone
408
409
<pre>
410
<code class="php">
411
#!/bin/sh
412
413
. scripts-config.sh
414
415
if [ $# -lt 1 ]; then
416
  echo "usage: $0 <site's folder name>"
417
  exit 1
418
fi
419
420
cd "$d6_sites_dir/$*"
421
pwd;
422
drush cron;
423
</code>
424
</pre>
425
426
h3. flush-files.sh
427
428
<pre>
429
<code class="php">
430
#!/bin/sh
431
432 66 Julien Enselme
help="# ARGS: number of files to keep, [email address]\n
433
# usage: flush files from a folder except a number equal to argument."
434 53 Julien Enselme
435
# Check if scripts-config.sh is imported.
436
if [ -z $scripts_utils ] ; then
437
    echo "Import of scripts-utils.sh required."
438 57 Julien Enselme
    . scripts-utils.sh
439 53 Julien Enselme
fi
440
441 66 Julien Enselme
check_arguments $# 1 "$help"
442 53 Julien Enselme
443
backups_number=`ls | wc -l`
444 60 Julien Enselme
let "number_of_backups_to_delete = backups_number - $1" > /dev/null
445 53 Julien Enselme
446 1 Julien Enselme
if [ $number_of_backups_to_delete -gt 0 ] ; then
447
    ls | head -$number_of_backups_to_delete | xargs rm
448 53 Julien Enselme
else
449 60 Julien Enselme
    if [ ! -z $2 ] ; then
450
        dir=`pwd`
451
        echo "There are not enough files in $dir to Flush it. Check if backup script works fine." | mail -s "[db] $dir has a backup problem" $2
452
    fi
453 53 Julien Enselme
fi
454
</code>
455
</pre>
456
457
h3. flush-full-databases-backups.sh
458
459
<pre>
460
<code class="php">
461
#!/bin/sh
462
463
. scripts-config.sh
464
465
# ARGS: Drupal version
466
467
cd $dir_full_backup/$1
468
469
for dir in `ls` ; do
470
    cd $dir
471
    flush-files.sh $db_full_backup_number $email_multi_assos
472
    cd -
473
done
474
</code>
475
</pre>
476
477
h3. flush-individual-databases-backups.sh
478
479
<pre>
480
<code class="php">
481
#!/bin/sh
482
483
. scripts-config.sh
484
485
# ARGS: manual or auto, Drupal version
486
487
cd $dir_individual_backup/$1/$2
488
489
for dir in `ls` ; do
490
    cd $dir
491
    if [ $1 = 'auto' ] ; then
492
        flush-files.sh $db_individual_auto_backup_number $email_multi_assos
493
    else
494 61 Julien Enselme
        flush-files.sh $db_individual_manual_backup_number
495 53 Julien Enselme
    fi
496
    cd -
497
done
498
</code>
499
</pre>
500
501
h2. Scripts Drupal 6
502
503
h3. d6-all-drush.sh
504
505
<pre>
506
<code class="php">
507
#!/bin/sh
508
509
. scripts-config.sh
510 21 Florent Torregrosa
. scripts-utils.sh
511
512 66 Julien Enselme
help="# ARGS: drush_command"
513 21 Florent Torregrosa
514 66 Julien Enselme
check_arguments $# 1 "$help"
515 21 Florent Torregrosa
516
for dir in `find $d6_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort` ; do
517
    cd $dir
518
    echo $dir
519
    drush $*
520 1 Julien Enselme
done
521 21 Florent Torregrosa
</code>
522 45 Florent Torregrosa
</pre>
523 21 Florent Torregrosa
524 53 Julien Enselme
h3. d6-all-dump-full.sh
525 45 Florent Torregrosa
526 37 Florent Torregrosa
<pre>
527
<code class="php">
528
#!/bin/sh
529
530
. scripts-config.sh
531
532
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
533
534
# Dump D6 database with webassos user.
535
mysqldump --single-transaction webassos -h myweb.serv.int -u webassos --password=password > $d6_dir_full_backup/webassos/$current_date.webassos.sql
536 41 Florent Torregrosa
537 1 Julien Enselme
# Dump D6 form database with forum user.
538 37 Florent Torregrosa
mysqldump forum -h myweb.serv.int -u forum --password=password > $d6_dir_full_backup/forum/$current_date.forum.sql
539
</code>
540 39 Florent Torregrosa
</pre>
541 1 Julien Enselme
542 53 Julien Enselme
h3. d6-all-reset-variables.sh
543 21 Florent Torregrosa
544
<pre>
545
<code class="php">
546
#!/bin/sh
547
548
. scripts-config.sh
549
550
cd $d6_dir_sites
551 1 Julien Enselme
d6-all-drush.sh vset --always-set reverse_proxy TRUE
552 21 Florent Torregrosa
d6-all-drush.sh vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
553
drush @sites -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
554
</code>
555
</pre>
556
557 53 Julien Enselme
h3. d6-all-update-contrib.sh
558 21 Florent Torregrosa
559
<pre>
560
<code class="php">
561
#!/bin/sh
562
563
. scripts-config.sh
564
565
d6-all-drush.sh -y en update
566 1 Julien Enselme
# Launch cron to check for updates.
567 21 Florent Torregrosa
d6-all-drush.sh -q cron
568
569
d6-all-drush.sh cc all
570
d6-all-dump-full.sh
571
572
d6-all-drush.sh -y upc --no-core
573 1 Julien Enselme
d6-all-drush.sh -y updb
574
575
d6-all-drush.sh -y dis update
576
577
d6-all-drush.sh cron
578
</code>
579
</pre>
580 41 Florent Torregrosa
581 53 Julien Enselme
h3. d6-dump-individual.sh
582 41 Florent Torregrosa
583
<pre>
584
<code class="php">
585
#!/bin/sh
586
587
. scripts-config.sh
588 21 Florent Torregrosa
589
# ARGS: auto or manual, site_prefix
590
# WARNING : backup are regularly flushed, put your backup in a safe place.
591
592
cd $dir_individual_backup
593 1 Julien Enselme
594
#!/bin/sh
595
596
. scripts-config.sh
597 21 Florent Torregrosa
598 1 Julien Enselme
# ARGS: auto or manual, site_prefix
599 21 Florent Torregrosa
# WARNING : backup are regularly flushed, put your backup in a safe place.
600
601
cd $dir_individual_backup
602
603 41 Florent Torregrosa
#récupération des tables du site dans le fichier liste_tables.temp
604
tables='_%'
605
liste="$1$tables"
606
607 21 Florent Torregrosa
mysql -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY -BNe "show tables like '"$liste"'" webassos | tr '\r\n' ' ' > liste_tables.temp
608
609
#transformation de cette liste en une variable
610
var=$(cat liste_tables.temp)
611
612 1 Julien Enselme
#sauvegarde de toutes ces tables
613 21 Florent Torregrosa
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
614
suffixe="_dump$current_date.sql"
615
fichier="$1$suffixe"
616
617 1 Julien Enselme
if [ $1 = 'auto' ] ; then
618 21 Florent Torregrosa
    mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_auto_backup/$fichier
619
else
620 1 Julien Enselme
    mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_manual_backup/$fichier
621 21 Florent Torregrosa
fi
622 1 Julien Enselme
623
#suppression du fichier temporaire utilisé
624 21 Florent Torregrosa
rm liste_tables.temp
625
</code>
626
</pre>
627
628 53 Julien Enselme
h2. Scripts Drupal 7
629 30 Florent Torregrosa
630 53 Julien Enselme
h3. d7-all-drush-logged.sh
631
632 1 Julien Enselme
<pre>
633 30 Florent Torregrosa
<code class="php">
634
#!/bin/sh
635
636 21 Florent Torregrosa
. scripts-config.sh
637 30 Florent Torregrosa
. scripts-utils.sh
638
639 66 Julien Enselme
help="# ARGS: drush_command"
640 21 Florent Torregrosa
641 66 Julien Enselme
check_arguments $# 1 "$help"
642 1 Julien Enselme
643 21 Florent Torregrosa
cd $d7_dir_sites
644 1 Julien Enselme
645 21 Florent Torregrosa
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
646
    cd $dir
647
    current_date=`date "+%Y-%m-%d_%Hh:%Mm"`
648
    echo "======$current_date======" >> $d7_dir_log/$dir.log
649 1 Julien Enselme
    echo "Arguments are: $* " >> $d7_dir_log/$dir.log
650 21 Florent Torregrosa
    drush $* >> $d7_dir_log/$dir.log
651
    #To have the same caracters limit we needed 'end' that have 4 caracters, we used the Croate language for this ! LOL 
652
    echo -e "=============KRAJ=============\n" >> $d7_dir_log/$dir.log
653
    cd -
654
done
655
</code>
656
</pre>
657 41 Florent Torregrosa
658 53 Julien Enselme
h3. d7-all-drush.sh
659 21 Florent Torregrosa
660
<pre>
661
<code class="php">
662
#!/bin/sh
663
664
. scripts-config.sh
665 1 Julien Enselme
. scripts-utils.sh
666 21 Florent Torregrosa
667 66 Julien Enselme
help="# ARGS: drush_command"
668 21 Florent Torregrosa
669 66 Julien Enselme
check_arguments $# 1 "$help"
670 21 Florent Torregrosa
671
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
672
    cd $dir
673
    echo $dir
674
    drush $*
675
done
676
</code>
677
</pre>
678
679 53 Julien Enselme
h3. d7-all-dump-full.sh
680 21 Florent Torregrosa
681 45 Florent Torregrosa
<pre>
682 21 Florent Torregrosa
<code class="php">
683 1 Julien Enselme
#!/bin/sh
684 21 Florent Torregrosa
685
. scripts-config.sh
686
687
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
688
689
# Dump D7 databases at once.
690
mysqldump -h $db_server -u $db_user -ppassword --all-databases > $d7_dir_full_backup/myassos/$current_date.d7_full.sql
691 41 Florent Torregrosa
</code>
692
</pre>
693
694 53 Julien Enselme
h3. d7-all-dump-individual.sh
695 21 Florent Torregrosa
696
<pre>
697 41 Florent Torregrosa
<code class="php">
698
#!/bin/sh
699
700 21 Florent Torregrosa
. scripts-config.sh
701 62 Julien Enselme
. scripts-utils.sh
702 21 Florent Torregrosa
703 66 Julien Enselme
help="# ARGS: auto or manual"
704 1 Julien Enselme
705 66 Julien Enselme
check_arguments $# 1 "$help"
706 21 Florent Torregrosa
707
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
708
709
cd $d7_dir_sites
710
711
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | cut -c3-` ; do
712 1 Julien Enselme
    cd $dir
713 21 Florent Torregrosa
    drush cc all
714
    if [ $1 = 'auto' ] ; then
715
        drush sql-dump --result-file="$d7_dir_individual_auto_backup/$dir/$current_date.$dir.sql"
716
    else
717 1 Julien Enselme
        drush sql-dump --result-file="$d7_dir_individual_manual_backup/$dir/$current_date.$dir.sql"
718 21 Florent Torregrosa
    fi
719
    cd -
720
done
721
</code>
722
</pre>
723
724 53 Julien Enselme
h3. d7-all-releases-info.sh
725 21 Florent Torregrosa
726
<pre>
727
<code class="php">
728
#!/bin/sh
729 1 Julien Enselme
730
. scripts-config.sh
731
732
# List version of a module or theme that exists in sites/all/*
733
# and all versions that are available for same project on drupal.org
734
735
# modules
736
cd $d7_dir_sites/all/modules
737
for dir in `ls -1` ; do
738
    if [ -d $dir ] ; then
739
        drush pm-releases $dir
740
    fi
741
done
742
743
# themes
744
cd $d7_dir_sites/all/themes
745
for dir in `ls -1` ; do
746
    if [ -d $dir ] ; then
747
        drush pm-releases $dir
748
    fi
749 21 Florent Torregrosa
done
750
</code>
751
</pre>
752 41 Florent Torregrosa
753 53 Julien Enselme
h3. d7-all-reset-variables.sh
754 41 Florent Torregrosa
755 1 Julien Enselme
<pre>
756 41 Florent Torregrosa
<code class="php">
757
#!/bin/sh
758
759
. scripts-config.sh
760
761
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
762
    cd $dir
763
    echo $dir
764
    d7-reset-variables.sh
765
done
766
</code>
767 1 Julien Enselme
</pre>
768 41 Florent Torregrosa
769 53 Julien Enselme
h3. d7-all-security-review.sh
770 41 Florent Torregrosa
771
<pre>
772
<code class="php">
773 1 Julien Enselme
#!/bin/sh
774 21 Florent Torregrosa
775
. scripts-config.sh
776
777
# grep -v success to keep site_name and error lines.
778 59 Florent Torregrosa
bash -c "d7-all-drush.sh security-review --store |& grep -v success | mail -s d7-all-security-review.sh $email_multi_assos"
779 21 Florent Torregrosa
</code>
780
</pre>
781
782 53 Julien Enselme
h3. d7-all-update-contrib.sh
783 21 Florent Torregrosa
784
<pre>
785
<code class="php">
786
#!/bin/sh
787
788
. scripts-config.sh
789 1 Julien Enselme
790 21 Florent Torregrosa
d7-all-drush.sh -y en update
791
# Launch cron to check for updates.
792
d7-all-drush.sh -q cron
793
794
d7-all-dump-individual.sh auto
795
796 1 Julien Enselme
d7-all-drush-logged.sh -y upc --no-core
797 21 Florent Torregrosa
d7-all-drush.sh -y updb
798
799
d7-all-drush.sh -y dis update
800
801
d7-all-drush.sh cron
802
</code>
803 1 Julien Enselme
</pre>
804 21 Florent Torregrosa
805 53 Julien Enselme
h3. d7-all-update-localisation.sh
806 21 Florent Torregrosa
807
<pre>
808
<code class="php">
809
#!/bin/sh
810
811
. scripts-config.sh
812
813
d7-all-drush.sh -y en l10n_update
814
d7-all-drush.sh l10n-update-refresh
815
d7-all-drush.sh l10n-update
816
d7-all-drush.sh -y dis l10n_update
817
</code>
818
</pre>
819 41 Florent Torregrosa
820 53 Julien Enselme
h3. d7-create-site.sh
821 1 Julien Enselme
822 21 Florent Torregrosa
<pre>
823
<code class="php">
824
#!/bin/sh
825 1 Julien Enselme
826 21 Florent Torregrosa
. scripts-config.sh
827
. scripts-config-site.sh $1
828
. scripts-utils.sh
829
830 66 Julien Enselme
help="# ARGS: site name"
831 21 Florent Torregrosa
832
######## Exceptions
833 66 Julien Enselme
check_arguments $# 1 "$help"
834 21 Florent Torregrosa
835
# "-" is forbidden because it provokes database error.
836
if [ `echo $1 | grep -` ] ; then
837
    echo '"-" is forbidden in the site name'
838
    exit 1
839
fi
840
841
# Site name length must be lower or equal to 16 due to database limitations.
842
if [`echo $1 | wc -c` -gt 16 ] ; then
843 41 Florent Torregrosa
    echo "site name can't have more than 16 characters"
844 21 Florent Torregrosa
    exit 1
845
fi
846
847 1 Julien Enselme
###### Initialisation
848 21 Florent Torregrosa
cd $d7_dir
849
db_password=`ask_password_db $db_server $db_user`
850
site_password=`generate_password`
851
852
# Check if site database already exists.
853
if mysql -h $db_server -u $db_user -e "USE $d7_site_name" -p$db_password 2>/dev/null ; then
854
    echo "Database $d7_site_name already exists"
855
    exit 1
856
fi
857
858
# Check if site folder already exists.
859
if [ -d $d7_site_dir ] ; then
860
    echo "Folder $d7_site_dir already exists"
861
    exit 1
862
fi
863
864
###### Main
865
mkdir $d7_site_dir
866
dir_site_name="assos.centrale-marseille.fr.$d7_site_name"
867
868
# Backup requirements
869 1 Julien Enselme
mkdir $d7_dir_individual_auto_backup/$dir_site_name
870 21 Florent Torregrosa
mkdir $d7_dir_individual_manual_backup/$dir_site_name
871
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
872
873
# NB : ls sort by considering the 1st characters
874
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql
875
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql2
876
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql3
877 1 Julien Enselme
878 21 Florent Torregrosa
# Create and grant privileges on database
879
mysql -h $db_server -u $db_user -e "CREATE DATABASE $d7_site_name" -p$db_password
880
mysql -h $db_server -u $db_user -e "GRANT ALL PRIVILEGES ON $d7_site_name.* TO '$d7_site_name'@'%' IDENTIFIED BY '$site_password'" -p$db_password
881
882
# Create settings.php
883
sed "s/\%\%DBUSER\%\%/$d7_site_name/ ; s/\%\%DBNAME\%\%/$d7_site_name/ ; s/\%\%DBPASS\%\%/$site_password/ ; s/\%\%nomsite\%\%/$d7_site_name/" < $d7_settings_template > $d7_site_settings
884
885
# Create symbolic link
886
cd $d7_dir
887
ln -s . $d7_site_name
888
889
# Next Instructions
890
echo "Go to http://assos.centrale-marseille.fr/$d7_site_name/install.php to continue."
891
echo "Press enter when ready to go on."
892 1 Julien Enselme
read key
893 41 Florent Torregrosa
894 21 Florent Torregrosa
# Init variables
895
d7-reset-variables.sh $d7_site_name
896
897
# Permissions
898
chmod -R 755 $d7_site_dir
899
chmod 400 $d7_site_settings
900
901
# Last instructions
902
echo "Last instructions:"
903
echo "- Advice the webmaster to close account creation on the website"
904
echo "- Give the webmaster a link to the club Drupal's tutorials "
905
echo "- Create a node of type \"Site\" on default"
906
echo "- Register the webmaster on webmasters@listes.centrale-marseille.fr"
907
</code>
908
</pre>
909
910 53 Julien Enselme
h3. d7-delete-site.sh
911 21 Florent Torregrosa
912
<pre>
913
<code class="php">
914
#!/bin/sh
915
916
. scripts-config.sh
917
. scripts-config-site.sh $1
918
. scripts-utils.sh
919
920 66 Julien Enselme
help="# ARGS: site name."
921 21 Florent Torregrosa
922 66 Julien Enselme
check_arguments $# 1 "$help"
923 21 Florent Torregrosa
924
db_password=`ask_password_db $db_server $db_user`
925
926
# Delete database.
927
mysql -h $db_server -u $db_user -p$db_password -e "DROP DATABASE $d7_site_name"
928
929
# Delete MYSQL user.
930 1 Julien Enselme
mysql -h $db_server -u $db_user -p$db_password -e "DROP USER '$d7_site_name'@'%'"
931 21 Florent Torregrosa
932
# Delete site's folder.
933
chmod -R 700 $d7_site_dir
934
rm -r $d7_site_dir
935
936 1 Julien Enselme
# Delete symbolic link.
937 21 Florent Torregrosa
rm $d7_dir/$d7_site_name
938
939
# Delete database backups.
940 1 Julien Enselme
rm -r $d7_dir_individual_auto_backup/assos.centrale-marseille.fr.$d7_site_name
941
rm -r $d7_dir_individual_manual_backup/assos.centrale-marseille.fr.$d7_site_name
942 21 Florent Torregrosa
943
echo "Don't forget to:"
944
echo "- Refresh node site on default"
945
echo "- Check particular behavior (normally detailed on site node)"
946
</code>
947
</pre>
948
949 53 Julien Enselme
h3. d7-reset-variables.sh
950 1 Julien Enselme
951 21 Florent Torregrosa
<pre>
952
<code class="php">
953
#!/bin/sh
954
955
. scripts-config.sh
956
957
# ARGS: [site_name]
958
959
if [ ! -z $1 ] ; then
960
    . scripts-config-site.sh $1
961
    cd $d7_site_dir
962
fi
963 1 Julien Enselme
964 41 Florent Torregrosa
current_timestamp=`date "+%s"`
965 21 Florent Torregrosa
966 41 Florent Torregrosa
random_1=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
967
random_2=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
968 21 Florent Torregrosa
969
####### General variables
970
drush -y vset --always-set reverse_proxy TRUE
971
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
972
drush -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
973
drush -y vset error_level 0
974
drush -y vset dblog_row_limit 1000
975
976 1 Julien Enselme
###### Hidden captcha
977 41 Florent Torregrosa
drush -y en hidden_captcha
978 21 Florent Torregrosa
# Log wrong answers.
979
drush -y vset captcha_log_wrong_responses 1
980
# Use hidden captcha for all forms.
981
drush -y sqlq --db-prefix "UPDATE {captcha_points} SET module = 'hidden_captcha', captcha_type = 'Hidden CAPTCHA' WHERE module is NULL;"
982
# Flush captcha cache.
983
drush -y vdel captcha_placement_map_cache
984
# Randomely generate a math question as the label of the hidden captcha field.
985
drush -y vset hidden_captcha_label "$random_1 + $random_2"
986
987
988
####### Piwik
989
drush -y en piwik
990
drush -y vset piwik_site_id "101"
991 41 Florent Torregrosa
drush -y vset piwik_url_http "http://piwik.centrale-marseille.fr/"
992 21 Florent Torregrosa
drush -y vset piwik_url_https "https://piwik.centrale-marseille.fr/"
993
# Piwik cache.
994
drush -y vset piwik_cache 1
995
drush -y vset piwik_visibility_roles "1"
996
# Piwik is enable for everyone except the administrator
997
drush -y vset --format=json piwik_roles '[3]'
998
drush -y vset piwik_page_title_hierarchy 1
999
# Activate local search.
1000
drush -y vset piwik_site_search 1
1001
1002
1003
###### Security review
1004
# For untrusted roles:
1005 41 Florent Torregrosa
# 1: anonymous user
1006 21 Florent Torregrosa
# 2: authenticated user
1007
# 3: administrator
1008
drush -y en security_review
1009 11 Julien Enselme
drush -y ev "variable_set('security_review_untrusted_roles', array('1'));"
1010 54 Florent Torregrosa
# Used to initialise entries in the database schema.
1011
drush security-review --store
1012
# Security review can't check for files permissions on multi_assos if launched within the web interface.
1013
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '1', skiptime = $current_timestamp, skipuid = '1' WHERE reviewcheck IN ('file_perms');"
1014
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '0', skiptime = '0', skipuid = NULL WHERE reviewcheck NOT IN ('file_perms');"
1015 1 Julien Enselme
</code>
1016
</pre>