Projet

Général

Profil

Scripts et taches planifiees » Historique » Version 71

Julien Enselme, 06/08/2013 13:08
La sauvegarde des bases de données : correction des liens vers les scripts

1 1 Julien Enselme
Afin de gagner du temps et d'éviter les erreurs humaines, des scripts ont été écrits tout au long du projet. Certains sont même exécutés automatiquement toutes les semaines.
2
3 13 Florent Torregrosa
{{>toc}}
4 1 Julien Enselme
5
h1. Les tâches planifiées
6
7 26 Florent Torregrosa
h2. Crontab
8
9 3 Julien Enselme
Pour exécuter ces tâches régulièrement, on utilise le "crontab":http://fr.wikipedia.org/wiki/Crontab. Il s'agit d'un programme installé sur notre serveur.
10 1 Julien Enselme
11 3 Julien Enselme
Pour voir et modifier la liste des actions :
12
# se connecter au serveur : <code>ssh assos@sas1.centrale-marseille</code>
13
# taper la commande pour voir le crontab <code>crontab -l</code>
14 47 Julien Enselme
# taper la commande pour modifier le crontab <code>crontab -e</code>.
15 1 Julien Enselme
16 47 Julien Enselme
{{important(Ne pas effectuer cette opération avant de s'être renseigné sur "vi":http://fr.wikipedia.org/wiki/Vi (l'éditeur de texte utilisé qui n'est pas vraiment intuitif :p) et sur "la syntaxe du crontab":http://fr.wikipedia.org/wiki/Crontab#Modification !
17
Les scripts doivent si possible s’exécuter avant le club Drupal pour que les problèmes éventuels y soient réglés)}}
18
19 69 Julien Enselme
{{tip(
20
* Pour utiliser un autre éditeur que vi taper : export EDITOR=editor_name exemple : vim, nano.
21
* d7-all-update-contrib.sh fait des sauvegardes automatiques individuelles pour les sites drupal)}}
22 26 Florent Torregrosa
23
<pre>
24
<code class="php">
25 42 Florent Torregrosa
######## How to use
26
# mm hh jj MMM JJJ task
27
# minutes hour day_month month day_week
28
# for more informations : https://fr.wikipedia.org/wiki/Crontab#Notation
29
######## End how to
30
31 1 Julien Enselme
32
### Cleaning
33 67 Julien Enselme
00 5 * * * /users/guest/assos/bin/all-reset-permissions.sh > /dev/null 2>&1
34
00 6 * * * /users/guest/assos/bin/delete-temporary-files.sh > /dev/null 2>&1
35 1 Julien Enselme
36
37 67 Julien Enselme
### Traqu-heure
38
00 7 * * * /users/guest/assos/bin/traqu_heure/script-restitution.sh > /dev/null 2>&1
39 68 Florent Torregrosa
0 * * * *  /users/guest/assos/bin/traqu_heure/script-traque.sh > /dev/null 2>&1
40 67 Julien Enselme
41
42 1 Julien Enselme
### Databases
43
# Create backup
44 67 Julien Enselme
00 8 1,15 * * /users/guest/assos/bin/d7-all-dump-full.sh > /dev/null 2>&1
45 68 Florent Torregrosa
00 1 * * 2    /users/guest/assos/bin/d6-all-dump-full.sh > /dev/null 2>&1
46 43 Florent Torregrosa
47
# flush backup
48 67 Julien Enselme
00 8 2,16 * * /users/guest/assos/bin/flush-full-databases-backups.sh d7 > /dev/null 2>&1
49 68 Florent Torregrosa
00 22 * * 2   /users/guest/assos/bin/flush-full-databases-backups.sh d6 > /dev/null 2>&1
50
00 23 * * 2   /users/guest/assos/bin/flush-individual-databases-backups.sh auto d7 > /dev/null 2>&1
51
00 1 * * 3    /users/guest/assos/bin/flush-individual-databases-backups.sh manual d7 > /dev/null 2>&1
52 42 Florent Torregrosa
53 43 Florent Torregrosa
54 67 Julien Enselme
### Drupal 7
55 56 Julien Enselme
00 22 * * 3 /users/guest/assos/bin/d7-all-update-contrib.sh | mail -s "d7-all-update-contrib.sh" assos
56 68 Florent Torregrosa
00 1 * * 4  /users/guest/assos/bin/d7-all-update-localisation.sh | mail -s "d7-all-update-localisation.sh" assos
57
00 2 * * 4  /users/guest/assos/bin/d7-all-security-review.sh
58
00 3 * * 4  /users/guest/assos/bin/d7-all-reset-variables.sh > /dev/null 2>&1
59
00 4 * * 4  /users/guest/assos/bin/d7-all-releases-info.sh | mail -s "d7-all-releases-info.sh" assos
60 42 Florent Torregrosa
61 67 Julien Enselme
62 43 Florent Torregrosa
### Drupal 6
63 67 Julien Enselme
00 22 * * 6 /users/guest/assos/bin/d6-all-update-contrib.sh > /dev/null 2>&1
64 42 Florent Torregrosa
00 23 * * 6 /users/guest/assos/bin/d6-all-reset-variables.sh > /dev/null 2>&1
65 43 Florent Torregrosa
66 67 Julien Enselme
# Launch drush cron on fellowings
67 68 Florent Torregrosa
45 17 * * *  /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.ginfo > /dev/null 2>&1
68
*/15 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.clubfinance > /dev/null 2>&1
69
45 3 * * *   /users/guest/assos/bin/drushcronone default > /dev/null 2>&1
70
35 00 * * *  /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.cac13 > /dev/null 2>&1
71
*/45 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.expat > /dev/null 2>&1
72
*/10 * * * * /users/guest/assos/bin/drushcronone forum.centrale-marseille.fr > /dev/null 2>&1
73 67 Julien Enselme
74 1 Julien Enselme
00 22 * * 7 /users/guest/assos/bin/disk-usage.sh | mail -s "disk-usage.sh" assos 
75
</code>
76 25 Florent Torregrosa
</pre>
77 1 Julien Enselme
78 67 Julien Enselme
La liste des tâches effectuées régulièrement est ci-dessous.
79 3 Julien Enselme
80 25 Florent Torregrosa
h2. La réinitialisation des variables
81
82 1 Julien Enselme
Cette tâche consiste à réinitialiser certaines variables qui donnent des droits considérés comme trop permissifs donc dangereux aux administrateurs des sites ou nécessaire pour d'autres scripts tels que piwik ou security_review.
83
84 25 Florent Torregrosa
En voici quelques exemples :
85 1 Julien Enselme
86
<pre>
87
<code class="bash">
88 25 Florent Torregrosa
drush -y vset error_level 0
89
</code>
90 1 Julien Enselme
</pre>
91 21 Florent Torregrosa
92 25 Florent Torregrosa
Cette commande permet de ne pas afficher les messages d'erreurs. En effet, ils contiennent parfois des informations sensibles sur l'installation et ne doivent donc pas être divulgués à n'importe qui.
93
94 3 Julien Enselme
<pre>
95 1 Julien Enselme
<code class="php">
96
drush -y vset --always-set reverse_proxy TRUE
97 25 Florent Torregrosa
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
98 1 Julien Enselme
</code>
99 3 Julien Enselme
</pre>
100 1 Julien Enselme
101 70 Julien Enselme
Ces commandes permettent de déclarer à Drupal les serveurs proxy du CRI afin d'éviter qu'il ne répertorie tous les visiteurs comme ayant l'adresse des serveurs sus-cités. **Pas fini : voir http://assos.centrale-marseille.fr/lessive/content/t%C3%A2che/rendre-les-settingsphp-conformes-%C3%A0-la-version-712-de-drupal**
102 1 Julien Enselme
103 6 Julien Enselme
h2. La sauvegarde des bases de données
104 1 Julien Enselme
105 3 Julien Enselme
Sur les installations d6 et d7 : une fois par semaine
106 1 Julien Enselme
107 71 Julien Enselme
On utilise les [[Scripts_et_taches_planifiees#d6-all-dump-fullsh|d6-all-dump-full.sh]], [[Scripts_et_taches_planifiees#d6-dump-individualsh|d6-dump-individual.sh]], [[Scripts_et_taches_planifiees#d7-all-dump-fullsh|d7-all-dump-full.sh]] et [[Scripts_et_taches_planifiees#d7-all-dump-individualsh|d7-all-dump-individual.sh]].
108 1 Julien Enselme
109 3 Julien Enselme
h2. La purge des sauvegardes de bdd
110 1 Julien Enselme
111
Sur les installations d6 et d7 : toutes les semaines
112
113
Cette tâche utilise le script de [[Scripts_et_taches_planifiees#purge_des_sauvegardes.sh|purge des sauvegardes]] afin de libérer de l'espace disque en supprimant les sauvegardes de bdd les plus vieilles.
114
115
h1. Liste des scripts à disposition
116 21 Florent Torregrosa
117
Les scripts utilisés sont hébergés dans le répertoire bin du compte assos.
118
119 53 Julien Enselme
h2. Configuration et utilitaires
120 1 Julien Enselme
121 53 Julien Enselme
h3. scripts-config.sh
122
123
<pre>
124
<code class="php">
125
#!/bin/sh
126
127
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/users/guest/assos/bin
128
PATH=$PATH:/usr/guest/assos/bin
129
130
scripts_config='imported'
131
132
email_multi_assos="assos@centrale-marseille.fr"
133
134
dir_multi_assos='/users/guest/assos'
135
dir_scripts="$dir_multi_assos/bin"
136
dir_template="$dir_multi_assos/template"
137
138
###### Database
139
db_server="myassos.serv.int"
140
db_user="assos"
141
dir_log="$dir_multi_assos/log"
142
143
###### Backup
144
db_full_backup_number=3
145 63 Julien Enselme
db_individual_manual_backup_number=2
146 53 Julien Enselme
db_individual_auto_backup_number=3
147
148
dir_backup="$dir_multi_assos/backup"
149
dir_full_backup="$dir_backup/full"
150
dir_individual_backup="$dir_backup/individual"
151 1 Julien Enselme
dir_individual_manual_backup="$dir_individual_backup/manual"
152
dir_individual_auto_backup="$dir_individual_backup/auto"
153
154
155
###### D7 variables
156
d7_dir="$dir_multi_assos/htmltest"
157
d7_dir_sites="$d7_dir/sites"
158
159
### Log
160
d7_dir_log="$dir_log/d7"
161
162
### Backup
163
d7_dir_full_backup="$dir_full_backup/d7"
164
d7_dir_individual_manual_backup="$dir_individual_manual_backup/d7"
165
d7_dir_individual_auto_backup="$dir_individual_auto_backup/d7"
166
167
### Template
168
d7_settings_template_name="settings-D7-bddinde-template.php"
169
d7_settings_template="$dir_template/$d7_settings_template_name"
170
171
172
###### D6 variables
173
d6_dir="$dir_multi_assos/html"
174
d6_dir_sites="$d6_dir/sites"
175
176
### Backup
177
d6_dir_full_backup="$dir_full_backup/d6"
178
d6_dir_individual_manual_backup="$dir_individual_manual_backup/d6"
179
d6_dir_individual_auto_backup="$dir_individual_auto_backup/d6"
180 64 Julien Enselme
</code>
181
</pre>
182
183
h3. scripts-config-site.sh
184
185
<pre>
186
<code class="php">
187
#!/bin/sh
188
189
# Sets useful variables of a site.
190
# ARGS: site_name
191
# Requires scripts-config.sh
192
193
# Check if scripts-config.sh is imported.
194
if [ -z $scripts_config ] ; then
195
    echo "Import of scripts-config.sh required."
196
    exit 1
197
fi
198
199
if [ -z $1 ] ; then
200
    echo "This script needs a site name as argument."
201
    exit 1
202
fi
203
204
scripts_config_site='imported'
205
206
d7_site_name=$1
207
d7_site_dir="$d7_dir_sites/assos.centrale-marseille.fr.$d7_site_name"
208
d7_site_settings="$d7_site_dir/settings.php"
209 53 Julien Enselme
</code>
210
</pre>
211
212
h3. scripts-utils.sh
213
214
<pre>
215
<code class="php">
216
#!/bin/sh
217
218
# This script contains useful functions for other scripts.
219
220
# Check if scripts-config.sh is imported.
221
if [ -z $scripts_config ] ; then
222
    echo "Import of scripts-config.sh required."
223 58 Julien Enselme
    . scripts-config.sh
224 53 Julien Enselme
fi
225
226
scripts_utils='imported'
227
228
ask_password() {
229
    # read -s doesn't work with sh.
230
    # usage: pass=`ask_password "password please:"`
231
    echo $1 >&2
232
    echo -n ">" >&2
233
    stty_avant=`stty -g`
234
    stty -echo
235
    read password
236
    stty $stty_avant
237
    echo "$password"
238
    unset password
239
}
240
241
ask_password_db() {
242
    # ARGS: server_name, user_name
243
    local db_password="pour_boucler"
244
    # empty db request to validate password
245
    while ! mysql -h $1 -u $2 -p$db_password -e "" 2>/dev/null ; do
246
        db_password=`ask_password "database password:"`
247
    done
248
    echo $db_password
249
}
250
251
generate_password(){
252
    # ARGS: [password_length]
253
    # The password contains special characters. '/' must be excluded to avoid sed malfunction.
254
255
    local site_password='/'
256
257
    if [ -z $1 ] ; then
258
        local password_length=20
259
    else
260
        local password_length=$1
261
    fi
262
263
    while echo "$site_password" | grep -Fq '/' ; do
264
        site_password=`dd if=/dev/urandom count=1 | uuencode -m - | head -n 2 | tail -n 1 | cut -c-$password_length`
265
    done
266
267
    echo $site_password
268
}
269
270
count_d7_sites(){
271
    find $d7_dir_sites -type d ! -name all -maxdepth 1 | wc -l
272
}
273
274
check_arguments() {
275 65 Julien Enselme
    # ARGS: number of arguments passed to script, number of arguments required, [help text]
276 53 Julien Enselme
    if [ $1 -lt $2  ] ; then
277 1 Julien Enselme
        echo "Number of arguments insuffisant."
278 65 Julien Enselme
	echo $3
279 53 Julien Enselme
        exit 1
280
    fi
281
}
282
</code>
283
</pre>
284
285
h2. Scripts généraux
286
287
h3. all-reset-permissions.sh
288
289
<pre>
290
<code class="php">
291
#!/bin/sh
292
293 21 Florent Torregrosa
. scripts-config.sh
294
295
# This script puts the correct permissions to sites folders, settings.php and scripts.
296
297
######### drupal 6
298
for dir in `find $d6_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all ! -name images ! -name languages` ; do
299
    chmod 755 $dir
300
    chmod 400 $dir/settings.php
301 35 Florent Torregrosa
done
302
303
######### drupal 7
304
for dir in `find $d7_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all` ; do
305
    chmod 755 $dir
306
    chmod 400 $dir/settings.php
307 21 Florent Torregrosa
done
308
309 1 Julien Enselme
####### bin
310 21 Florent Torregrosa
chmod -R 700 $dir_scripts
311
312
####### backup
313
chmod -R 700 $dir_backup
314
315
####### log
316
chmod -R 700 $dir_log
317
</code>
318 1 Julien Enselme
</pre>
319 21 Florent Torregrosa
320 53 Julien Enselme
h3. contrib-usage.sh
321 29 Florent Torregrosa
322 21 Florent Torregrosa
<pre>
323
<code class="php">
324
#!/bin/sh
325
326
. scripts-config.sh
327
. scripts-utils.sh
328
329 66 Julien Enselme
help="# ARGS: Drupal_version, project_status, project_name\n
330
# List sites that have the project_name with the corresponding project_status.\n
331
# project status: enabled or disabled"
332 21 Florent Torregrosa
333 66 Julien Enselme
check_arguments $# 3 "$help"
334 21 Florent Torregrosa
335
if [ $1 = d6 ] ; then
336
    cd $d6_dir_sites
337
else
338
    cd $d7_dir_sites
339
fi
340 29 Florent Torregrosa
341 21 Florent Torregrosa
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort ` ; do
342
    # List non-core projects that correspond to the status.
343
    # Keep project_name if listed.
344
    # Count line result. 0 if not listed or 1 if listed.
345 1 Julien Enselme
	# Print site_dir if listed.
346 21 Florent Torregrosa
347
    cd $dir;
348
	if [ 1 -eq `drush pml --no-core --status=$2 | grep $3 | wc -l` ] ; then
349
        echo $dir; 
350
    fi
351
    cd -
352
done
353 1 Julien Enselme
</code>
354 21 Florent Torregrosa
</pre>
355
356 53 Julien Enselme
h3. delete-temporary-files.sh
357 21 Florent Torregrosa
358 41 Florent Torregrosa
<pre>
359 21 Florent Torregrosa
<code class="php">
360 1 Julien Enselme
#!/bin/sh
361
362
. scripts-config.sh
363 53 Julien Enselme
364
find $dir_multi_assos -name '*~' -delete
365
</code>
366
</pre>
367
368
h3. disk-usage.sh
369
370
<pre>
371
<code class="php">
372
#!/bin/sh
373
374
. scripts-config.sh
375
376
echo -e "Quotas: 9.1 GB\n"
377
378
echo "Assos:"
379
du -hcs $dir_multi_assos | grep total
380
echo -e "\n"
381
382
echo "D6:"
383
cd $d6_dir_sites
384
du -h -d 1
385
echo -e "\n"
386
387
echo "D7:"
388
cd $d7_dir_sites
389
du -h -d 1
390
echo -e "\n"
391
392
echo "Logs:"
393
du -hcs $dir_log | grep total
394
echo -e "\n"
395
396
echo "Backups:"
397
cd $dir_backup
398
du -h -d 2
399
</code>
400
</pre>
401
402
h3. drushcronone
403
404
<pre>
405
<code class="php">
406
#!/bin/sh
407
408
. scripts-config.sh
409
410
if [ $# -lt 1 ]; then
411
  echo "usage: $0 <site's folder name>"
412
  exit 1
413
fi
414
415
cd "$d6_sites_dir/$*"
416
pwd;
417
drush cron;
418
</code>
419
</pre>
420
421
h3. flush-files.sh
422
423
<pre>
424
<code class="php">
425
#!/bin/sh
426
427 66 Julien Enselme
help="# ARGS: number of files to keep, [email address]\n
428
# usage: flush files from a folder except a number equal to argument."
429 53 Julien Enselme
430
# Check if scripts-config.sh is imported.
431
if [ -z $scripts_utils ] ; then
432
    echo "Import of scripts-utils.sh required."
433 57 Julien Enselme
    . scripts-utils.sh
434 53 Julien Enselme
fi
435
436 66 Julien Enselme
check_arguments $# 1 "$help"
437 53 Julien Enselme
438
backups_number=`ls | wc -l`
439 60 Julien Enselme
let "number_of_backups_to_delete = backups_number - $1" > /dev/null
440 53 Julien Enselme
441 1 Julien Enselme
if [ $number_of_backups_to_delete -gt 0 ] ; then
442
    ls | head -$number_of_backups_to_delete | xargs rm
443 53 Julien Enselme
else
444 60 Julien Enselme
    if [ ! -z $2 ] ; then
445
        dir=`pwd`
446
        echo "There are not enough files in $dir to Flush it. Check if backup script works fine." | mail -s "[db] $dir has a backup problem" $2
447
    fi
448 53 Julien Enselme
fi
449
</code>
450
</pre>
451
452
h3. flush-full-databases-backups.sh
453
454
<pre>
455
<code class="php">
456
#!/bin/sh
457
458
. scripts-config.sh
459
460
# ARGS: Drupal version
461
462
cd $dir_full_backup/$1
463
464
for dir in `ls` ; do
465
    cd $dir
466
    flush-files.sh $db_full_backup_number $email_multi_assos
467
    cd -
468
done
469
</code>
470
</pre>
471
472
h3. flush-individual-databases-backups.sh
473
474
<pre>
475
<code class="php">
476
#!/bin/sh
477
478
. scripts-config.sh
479
480
# ARGS: manual or auto, Drupal version
481
482
cd $dir_individual_backup/$1/$2
483
484
for dir in `ls` ; do
485
    cd $dir
486
    if [ $1 = 'auto' ] ; then
487
        flush-files.sh $db_individual_auto_backup_number $email_multi_assos
488
    else
489 61 Julien Enselme
        flush-files.sh $db_individual_manual_backup_number
490 53 Julien Enselme
    fi
491
    cd -
492
done
493
</code>
494
</pre>
495
496
h2. Scripts Drupal 6
497
498
h3. d6-all-drush.sh
499
500
<pre>
501
<code class="php">
502
#!/bin/sh
503
504
. scripts-config.sh
505 21 Florent Torregrosa
. scripts-utils.sh
506
507 66 Julien Enselme
help="# ARGS: drush_command"
508 21 Florent Torregrosa
509 66 Julien Enselme
check_arguments $# 1 "$help"
510 21 Florent Torregrosa
511
for dir in `find $d6_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort` ; do
512
    cd $dir
513
    echo $dir
514
    drush $*
515 1 Julien Enselme
done
516 21 Florent Torregrosa
</code>
517 45 Florent Torregrosa
</pre>
518 21 Florent Torregrosa
519 53 Julien Enselme
h3. d6-all-dump-full.sh
520 45 Florent Torregrosa
521 37 Florent Torregrosa
<pre>
522
<code class="php">
523
#!/bin/sh
524
525
. scripts-config.sh
526
527
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
528
529
# Dump D6 database with webassos user.
530
mysqldump --single-transaction webassos -h myweb.serv.int -u webassos --password=password > $d6_dir_full_backup/webassos/$current_date.webassos.sql
531 41 Florent Torregrosa
532 1 Julien Enselme
# Dump D6 form database with forum user.
533 37 Florent Torregrosa
mysqldump forum -h myweb.serv.int -u forum --password=password > $d6_dir_full_backup/forum/$current_date.forum.sql
534
</code>
535 39 Florent Torregrosa
</pre>
536 1 Julien Enselme
537 53 Julien Enselme
h3. d6-all-reset-variables.sh
538 21 Florent Torregrosa
539
<pre>
540
<code class="php">
541
#!/bin/sh
542
543
. scripts-config.sh
544
545
cd $d6_dir_sites
546 1 Julien Enselme
d6-all-drush.sh vset --always-set reverse_proxy TRUE
547 21 Florent Torregrosa
d6-all-drush.sh vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
548
drush @sites -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
549
</code>
550
</pre>
551
552 53 Julien Enselme
h3. d6-all-update-contrib.sh
553 21 Florent Torregrosa
554
<pre>
555
<code class="php">
556
#!/bin/sh
557
558
. scripts-config.sh
559
560
d6-all-drush.sh -y en update
561 1 Julien Enselme
# Launch cron to check for updates.
562 21 Florent Torregrosa
d6-all-drush.sh -q cron
563
564
d6-all-drush.sh cc all
565
d6-all-dump-full.sh
566
567
d6-all-drush.sh -y upc --no-core
568 1 Julien Enselme
d6-all-drush.sh -y updb
569
570
d6-all-drush.sh -y dis update
571
572
d6-all-drush.sh cron
573
</code>
574
</pre>
575 41 Florent Torregrosa
576 53 Julien Enselme
h3. d6-dump-individual.sh
577 41 Florent Torregrosa
578
<pre>
579
<code class="php">
580
#!/bin/sh
581
582
. scripts-config.sh
583 21 Florent Torregrosa
584
# ARGS: auto or manual, site_prefix
585
# WARNING : backup are regularly flushed, put your backup in a safe place.
586
587
cd $dir_individual_backup
588 1 Julien Enselme
589
#!/bin/sh
590
591
. scripts-config.sh
592 21 Florent Torregrosa
593 1 Julien Enselme
# ARGS: auto or manual, site_prefix
594 21 Florent Torregrosa
# WARNING : backup are regularly flushed, put your backup in a safe place.
595
596
cd $dir_individual_backup
597
598 41 Florent Torregrosa
#récupération des tables du site dans le fichier liste_tables.temp
599
tables='_%'
600
liste="$1$tables"
601
602 21 Florent Torregrosa
mysql -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY -BNe "show tables like '"$liste"'" webassos | tr '\r\n' ' ' > liste_tables.temp
603
604
#transformation de cette liste en une variable
605
var=$(cat liste_tables.temp)
606
607 1 Julien Enselme
#sauvegarde de toutes ces tables
608 21 Florent Torregrosa
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
609
suffixe="_dump$current_date.sql"
610
fichier="$1$suffixe"
611
612 1 Julien Enselme
if [ $1 = 'auto' ] ; then
613 21 Florent Torregrosa
    mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_auto_backup/$fichier
614
else
615 1 Julien Enselme
    mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_manual_backup/$fichier
616 21 Florent Torregrosa
fi
617 1 Julien Enselme
618
#suppression du fichier temporaire utilisé
619 21 Florent Torregrosa
rm liste_tables.temp
620
</code>
621
</pre>
622
623 53 Julien Enselme
h2. Scripts Drupal 7
624 30 Florent Torregrosa
625 53 Julien Enselme
h3. d7-all-drush-logged.sh
626
627 1 Julien Enselme
<pre>
628 30 Florent Torregrosa
<code class="php">
629
#!/bin/sh
630
631 21 Florent Torregrosa
. scripts-config.sh
632 30 Florent Torregrosa
. scripts-utils.sh
633
634 66 Julien Enselme
help="# ARGS: drush_command"
635 21 Florent Torregrosa
636 66 Julien Enselme
check_arguments $# 1 "$help"
637 1 Julien Enselme
638 21 Florent Torregrosa
cd $d7_dir_sites
639 1 Julien Enselme
640 21 Florent Torregrosa
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
641
    cd $dir
642
    current_date=`date "+%Y-%m-%d_%Hh:%Mm"`
643
    echo "======$current_date======" >> $d7_dir_log/$dir.log
644 1 Julien Enselme
    echo "Arguments are: $* " >> $d7_dir_log/$dir.log
645 21 Florent Torregrosa
    drush $* >> $d7_dir_log/$dir.log
646
    #To have the same caracters limit we needed 'end' that have 4 caracters, we used the Croate language for this ! LOL 
647
    echo -e "=============KRAJ=============\n" >> $d7_dir_log/$dir.log
648
    cd -
649
done
650
</code>
651
</pre>
652 41 Florent Torregrosa
653 53 Julien Enselme
h3. d7-all-drush.sh
654 21 Florent Torregrosa
655
<pre>
656
<code class="php">
657
#!/bin/sh
658
659
. scripts-config.sh
660 1 Julien Enselme
. scripts-utils.sh
661 21 Florent Torregrosa
662 66 Julien Enselme
help="# ARGS: drush_command"
663 21 Florent Torregrosa
664 66 Julien Enselme
check_arguments $# 1 "$help"
665 21 Florent Torregrosa
666
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
667
    cd $dir
668
    echo $dir
669
    drush $*
670
done
671
</code>
672
</pre>
673
674 53 Julien Enselme
h3. d7-all-dump-full.sh
675 21 Florent Torregrosa
676 45 Florent Torregrosa
<pre>
677 21 Florent Torregrosa
<code class="php">
678 1 Julien Enselme
#!/bin/sh
679 21 Florent Torregrosa
680
. scripts-config.sh
681
682
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
683
684
# Dump D7 databases at once.
685
mysqldump -h $db_server -u $db_user -ppassword --all-databases > $d7_dir_full_backup/myassos/$current_date.d7_full.sql
686 41 Florent Torregrosa
</code>
687
</pre>
688
689 53 Julien Enselme
h3. d7-all-dump-individual.sh
690 21 Florent Torregrosa
691
<pre>
692 41 Florent Torregrosa
<code class="php">
693
#!/bin/sh
694
695 21 Florent Torregrosa
. scripts-config.sh
696 62 Julien Enselme
. scripts-utils.sh
697 21 Florent Torregrosa
698 66 Julien Enselme
help="# ARGS: auto or manual"
699 1 Julien Enselme
700 66 Julien Enselme
check_arguments $# 1 "$help"
701 21 Florent Torregrosa
702
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
703
704
cd $d7_dir_sites
705
706
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | cut -c3-` ; do
707 1 Julien Enselme
    cd $dir
708 21 Florent Torregrosa
    drush cc all
709
    if [ $1 = 'auto' ] ; then
710
        drush sql-dump --result-file="$d7_dir_individual_auto_backup/$dir/$current_date.$dir.sql"
711
    else
712 1 Julien Enselme
        drush sql-dump --result-file="$d7_dir_individual_manual_backup/$dir/$current_date.$dir.sql"
713 21 Florent Torregrosa
    fi
714
    cd -
715
done
716
</code>
717
</pre>
718
719 53 Julien Enselme
h3. d7-all-releases-info.sh
720 21 Florent Torregrosa
721
<pre>
722
<code class="php">
723
#!/bin/sh
724 1 Julien Enselme
725
. scripts-config.sh
726
727
# List version of a module or theme that exists in sites/all/*
728
# and all versions that are available for same project on drupal.org
729
730
# modules
731
cd $d7_dir_sites/all/modules
732
for dir in `ls -1` ; do
733
    if [ -d $dir ] ; then
734
        drush pm-releases $dir
735
    fi
736
done
737
738
# themes
739
cd $d7_dir_sites/all/themes
740
for dir in `ls -1` ; do
741
    if [ -d $dir ] ; then
742
        drush pm-releases $dir
743
    fi
744 21 Florent Torregrosa
done
745
</code>
746
</pre>
747 41 Florent Torregrosa
748 53 Julien Enselme
h3. d7-all-reset-variables.sh
749 41 Florent Torregrosa
750 1 Julien Enselme
<pre>
751 41 Florent Torregrosa
<code class="php">
752
#!/bin/sh
753
754
. scripts-config.sh
755
756
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
757
    cd $dir
758
    echo $dir
759
    d7-reset-variables.sh
760
done
761
</code>
762 1 Julien Enselme
</pre>
763 41 Florent Torregrosa
764 53 Julien Enselme
h3. d7-all-security-review.sh
765 41 Florent Torregrosa
766
<pre>
767
<code class="php">
768 1 Julien Enselme
#!/bin/sh
769 21 Florent Torregrosa
770
. scripts-config.sh
771
772
# grep -v success to keep site_name and error lines.
773 59 Florent Torregrosa
bash -c "d7-all-drush.sh security-review --store |& grep -v success | mail -s d7-all-security-review.sh $email_multi_assos"
774 21 Florent Torregrosa
</code>
775
</pre>
776
777 53 Julien Enselme
h3. d7-all-update-contrib.sh
778 21 Florent Torregrosa
779
<pre>
780
<code class="php">
781
#!/bin/sh
782
783
. scripts-config.sh
784 1 Julien Enselme
785 21 Florent Torregrosa
d7-all-drush.sh -y en update
786
# Launch cron to check for updates.
787
d7-all-drush.sh -q cron
788
789
d7-all-dump-individual.sh auto
790
791 1 Julien Enselme
d7-all-drush-logged.sh -y upc --no-core
792 21 Florent Torregrosa
d7-all-drush.sh -y updb
793
794
d7-all-drush.sh -y dis update
795
796
d7-all-drush.sh cron
797
</code>
798 1 Julien Enselme
</pre>
799 21 Florent Torregrosa
800 53 Julien Enselme
h3. d7-all-update-localisation.sh
801 21 Florent Torregrosa
802
<pre>
803
<code class="php">
804
#!/bin/sh
805
806
. scripts-config.sh
807
808
d7-all-drush.sh -y en l10n_update
809
d7-all-drush.sh l10n-update-refresh
810
d7-all-drush.sh l10n-update
811
d7-all-drush.sh -y dis l10n_update
812
</code>
813
</pre>
814 41 Florent Torregrosa
815 53 Julien Enselme
h3. d7-create-site.sh
816 1 Julien Enselme
817 21 Florent Torregrosa
<pre>
818
<code class="php">
819
#!/bin/sh
820 1 Julien Enselme
821 21 Florent Torregrosa
. scripts-config.sh
822
. scripts-config-site.sh $1
823
. scripts-utils.sh
824
825 66 Julien Enselme
help="# ARGS: site name"
826 21 Florent Torregrosa
827
######## Exceptions
828 66 Julien Enselme
check_arguments $# 1 "$help"
829 21 Florent Torregrosa
830
# "-" is forbidden because it provokes database error.
831
if [ `echo $1 | grep -` ] ; then
832
    echo '"-" is forbidden in the site name'
833
    exit 1
834
fi
835
836
# Site name length must be lower or equal to 16 due to database limitations.
837
if [`echo $1 | wc -c` -gt 16 ] ; then
838 41 Florent Torregrosa
    echo "site name can't have more than 16 characters"
839 21 Florent Torregrosa
    exit 1
840
fi
841
842 1 Julien Enselme
###### Initialisation
843 21 Florent Torregrosa
cd $d7_dir
844
db_password=`ask_password_db $db_server $db_user`
845
site_password=`generate_password`
846
847
# Check if site database already exists.
848
if mysql -h $db_server -u $db_user -e "USE $d7_site_name" -p$db_password 2>/dev/null ; then
849
    echo "Database $d7_site_name already exists"
850
    exit 1
851
fi
852
853
# Check if site folder already exists.
854
if [ -d $d7_site_dir ] ; then
855
    echo "Folder $d7_site_dir already exists"
856
    exit 1
857
fi
858
859
###### Main
860
mkdir $d7_site_dir
861
dir_site_name="assos.centrale-marseille.fr.$d7_site_name"
862
863
# Backup requirements
864 1 Julien Enselme
mkdir $d7_dir_individual_auto_backup/$dir_site_name
865 21 Florent Torregrosa
mkdir $d7_dir_individual_manual_backup/$dir_site_name
866
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
867
868
# NB : ls sort by considering the 1st characters
869
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql
870
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql2
871
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql3
872 1 Julien Enselme
873 21 Florent Torregrosa
# Create and grant privileges on database
874
mysql -h $db_server -u $db_user -e "CREATE DATABASE $d7_site_name" -p$db_password
875
mysql -h $db_server -u $db_user -e "GRANT ALL PRIVILEGES ON $d7_site_name.* TO '$d7_site_name'@'%' IDENTIFIED BY '$site_password'" -p$db_password
876
877
# Create settings.php
878
sed "s/\%\%DBUSER\%\%/$d7_site_name/ ; s/\%\%DBNAME\%\%/$d7_site_name/ ; s/\%\%DBPASS\%\%/$site_password/ ; s/\%\%nomsite\%\%/$d7_site_name/" < $d7_settings_template > $d7_site_settings
879
880
# Create symbolic link
881
cd $d7_dir
882
ln -s . $d7_site_name
883
884
# Next Instructions
885
echo "Go to http://assos.centrale-marseille.fr/$d7_site_name/install.php to continue."
886
echo "Press enter when ready to go on."
887 1 Julien Enselme
read key
888 41 Florent Torregrosa
889 21 Florent Torregrosa
# Init variables
890
d7-reset-variables.sh $d7_site_name
891
892
# Permissions
893
chmod -R 755 $d7_site_dir
894
chmod 400 $d7_site_settings
895
896
# Last instructions
897
echo "Last instructions:"
898
echo "- Advice the webmaster to close account creation on the website"
899
echo "- Give the webmaster a link to the club Drupal's tutorials "
900
echo "- Create a node of type \"Site\" on default"
901
echo "- Register the webmaster on webmasters@listes.centrale-marseille.fr"
902
</code>
903
</pre>
904
905 53 Julien Enselme
h3. d7-delete-site.sh
906 21 Florent Torregrosa
907
<pre>
908
<code class="php">
909
#!/bin/sh
910
911
. scripts-config.sh
912
. scripts-config-site.sh $1
913
. scripts-utils.sh
914
915 66 Julien Enselme
help="# ARGS: site name."
916 21 Florent Torregrosa
917 66 Julien Enselme
check_arguments $# 1 "$help"
918 21 Florent Torregrosa
919
db_password=`ask_password_db $db_server $db_user`
920
921
# Delete database.
922
mysql -h $db_server -u $db_user -p$db_password -e "DROP DATABASE $d7_site_name"
923
924
# Delete MYSQL user.
925 1 Julien Enselme
mysql -h $db_server -u $db_user -p$db_password -e "DROP USER '$d7_site_name'@'%'"
926 21 Florent Torregrosa
927
# Delete site's folder.
928
chmod -R 700 $d7_site_dir
929
rm -r $d7_site_dir
930
931 1 Julien Enselme
# Delete symbolic link.
932 21 Florent Torregrosa
rm $d7_dir/$d7_site_name
933
934
# Delete database backups.
935 1 Julien Enselme
rm -r $d7_dir_individual_auto_backup/assos.centrale-marseille.fr.$d7_site_name
936
rm -r $d7_dir_individual_manual_backup/assos.centrale-marseille.fr.$d7_site_name
937 21 Florent Torregrosa
938
echo "Don't forget to:"
939
echo "- Refresh node site on default"
940
echo "- Check particular behavior (normally detailed on site node)"
941
</code>
942
</pre>
943
944 53 Julien Enselme
h3. d7-reset-variables.sh
945 1 Julien Enselme
946 21 Florent Torregrosa
<pre>
947
<code class="php">
948
#!/bin/sh
949
950
. scripts-config.sh
951
952
# ARGS: [site_name]
953
954
if [ ! -z $1 ] ; then
955
    . scripts-config-site.sh $1
956
    cd $d7_site_dir
957
fi
958 1 Julien Enselme
959 41 Florent Torregrosa
current_timestamp=`date "+%s"`
960 21 Florent Torregrosa
961 41 Florent Torregrosa
random_1=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
962
random_2=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
963 21 Florent Torregrosa
964
####### General variables
965
drush -y vset --always-set reverse_proxy TRUE
966
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
967
drush -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
968
drush -y vset error_level 0
969
drush -y vset dblog_row_limit 1000
970
971 1 Julien Enselme
###### Hidden captcha
972 41 Florent Torregrosa
drush -y en hidden_captcha
973 21 Florent Torregrosa
# Log wrong answers.
974
drush -y vset captcha_log_wrong_responses 1
975
# Use hidden captcha for all forms.
976
drush -y sqlq --db-prefix "UPDATE {captcha_points} SET module = 'hidden_captcha', captcha_type = 'Hidden CAPTCHA' WHERE module is NULL;"
977
# Flush captcha cache.
978
drush -y vdel captcha_placement_map_cache
979
# Randomely generate a math question as the label of the hidden captcha field.
980
drush -y vset hidden_captcha_label "$random_1 + $random_2"
981
982
983
####### Piwik
984
drush -y en piwik
985
drush -y vset piwik_site_id "101"
986 41 Florent Torregrosa
drush -y vset piwik_url_http "http://piwik.centrale-marseille.fr/"
987 21 Florent Torregrosa
drush -y vset piwik_url_https "https://piwik.centrale-marseille.fr/"
988
# Piwik cache.
989
drush -y vset piwik_cache 1
990
drush -y vset piwik_visibility_roles "1"
991
# Piwik is enable for everyone except the administrator
992
drush -y vset --format=json piwik_roles '[3]'
993
drush -y vset piwik_page_title_hierarchy 1
994
# Activate local search.
995
drush -y vset piwik_site_search 1
996
997
998
###### Security review
999
# For untrusted roles:
1000 41 Florent Torregrosa
# 1: anonymous user
1001 21 Florent Torregrosa
# 2: authenticated user
1002
# 3: administrator
1003
drush -y en security_review
1004 11 Julien Enselme
drush -y ev "variable_set('security_review_untrusted_roles', array('1'));"
1005 54 Florent Torregrosa
# Used to initialise entries in the database schema.
1006
drush security-review --store
1007
# Security review can't check for files permissions on multi_assos if launched within the web interface.
1008
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '1', skiptime = $current_timestamp, skipuid = '1' WHERE reviewcheck IN ('file_perms');"
1009
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '0', skiptime = '0', skipuid = NULL WHERE reviewcheck NOT IN ('file_perms');"
1010 1 Julien Enselme
</code>
1011
</pre>