Projet

Général

Profil

Scripts et taches planifiees » Historique » Version 74

Florent Torregrosa, 06/08/2013 13:22
plus d'explications sur le tip -> important à propos de d7-all-update-contrib.sh dans le crontab

1 1 Julien Enselme
Afin de gagner du temps et d'éviter les erreurs humaines, des scripts ont été écrits tout au long du projet. Certains sont même exécutés automatiquement toutes les semaines.
2
3 13 Florent Torregrosa
{{>toc}}
4 1 Julien Enselme
5
h1. Les tâches planifiées
6
7 26 Florent Torregrosa
h2. Crontab
8
9 3 Julien Enselme
Pour exécuter ces tâches régulièrement, on utilise le "crontab":http://fr.wikipedia.org/wiki/Crontab. Il s'agit d'un programme installé sur notre serveur.
10 1 Julien Enselme
11 3 Julien Enselme
Pour voir et modifier la liste des actions :
12
# se connecter au serveur : <code>ssh assos@sas1.centrale-marseille</code>
13
# taper la commande pour voir le crontab <code>crontab -l</code>
14 47 Julien Enselme
# taper la commande pour modifier le crontab <code>crontab -e</code>.
15 1 Julien Enselme
16 47 Julien Enselme
{{important(Ne pas effectuer cette opération avant de s'être renseigné sur "vi":http://fr.wikipedia.org/wiki/Vi (l'éditeur de texte utilisé qui n'est pas vraiment intuitif :p) et sur "la syntaxe du crontab":http://fr.wikipedia.org/wiki/Crontab#Modification !
17
Les scripts doivent si possible s’exécuter avant le club Drupal pour que les problèmes éventuels y soient réglés)}}
18
19 69 Julien Enselme
{{tip(
20
* Pour utiliser un autre éditeur que vi taper : export EDITOR=editor_name exemple : vim, nano.
21 74 Florent Torregrosa
)}}
22
23
{{important(
24
* d7-all-update-contrib.sh fait des sauvegardes automatiques individuelles pour les sites Drupal 7 en faisant appel à d7-all-dump-indiviual.sh auto
25
* C'est pour cela que d7-all-dump-individual.sh auto n'est pas appelé dans le crontab.
26
)}}
27 26 Florent Torregrosa
28
<pre>
29
<code class="php">
30 42 Florent Torregrosa
######## How to use
31
# mm hh jj MMM JJJ task
32
# minutes hour day_month month day_week
33
# for more informations : https://fr.wikipedia.org/wiki/Crontab#Notation
34
######## End how to
35
36 1 Julien Enselme
37
### Cleaning
38 67 Julien Enselme
00 5 * * * /users/guest/assos/bin/all-reset-permissions.sh > /dev/null 2>&1
39
00 6 * * * /users/guest/assos/bin/delete-temporary-files.sh > /dev/null 2>&1
40 1 Julien Enselme
41
42 67 Julien Enselme
### Traqu-heure
43
00 7 * * * /users/guest/assos/bin/traqu_heure/script-restitution.sh > /dev/null 2>&1
44 68 Florent Torregrosa
0 * * * *  /users/guest/assos/bin/traqu_heure/script-traque.sh > /dev/null 2>&1
45 67 Julien Enselme
46
47 1 Julien Enselme
### Databases
48
# Create backup
49 67 Julien Enselme
00 8 1,15 * * /users/guest/assos/bin/d7-all-dump-full.sh > /dev/null 2>&1
50 68 Florent Torregrosa
00 1 * * 2    /users/guest/assos/bin/d6-all-dump-full.sh > /dev/null 2>&1
51 43 Florent Torregrosa
52
# flush backup
53 67 Julien Enselme
00 8 2,16 * * /users/guest/assos/bin/flush-full-databases-backups.sh d7 > /dev/null 2>&1
54 68 Florent Torregrosa
00 22 * * 2   /users/guest/assos/bin/flush-full-databases-backups.sh d6 > /dev/null 2>&1
55
00 23 * * 2   /users/guest/assos/bin/flush-individual-databases-backups.sh auto d7 > /dev/null 2>&1
56
00 1 * * 3    /users/guest/assos/bin/flush-individual-databases-backups.sh manual d7 > /dev/null 2>&1
57 42 Florent Torregrosa
58 43 Florent Torregrosa
59 67 Julien Enselme
### Drupal 7
60 56 Julien Enselme
00 22 * * 3 /users/guest/assos/bin/d7-all-update-contrib.sh | mail -s "d7-all-update-contrib.sh" assos
61 68 Florent Torregrosa
00 1 * * 4  /users/guest/assos/bin/d7-all-update-localisation.sh | mail -s "d7-all-update-localisation.sh" assos
62
00 2 * * 4  /users/guest/assos/bin/d7-all-security-review.sh
63
00 3 * * 4  /users/guest/assos/bin/d7-all-reset-variables.sh > /dev/null 2>&1
64
00 4 * * 4  /users/guest/assos/bin/d7-all-releases-info.sh | mail -s "d7-all-releases-info.sh" assos
65 42 Florent Torregrosa
66 67 Julien Enselme
67 43 Florent Torregrosa
### Drupal 6
68 67 Julien Enselme
00 22 * * 6 /users/guest/assos/bin/d6-all-update-contrib.sh > /dev/null 2>&1
69 42 Florent Torregrosa
00 23 * * 6 /users/guest/assos/bin/d6-all-reset-variables.sh > /dev/null 2>&1
70 43 Florent Torregrosa
71 67 Julien Enselme
# Launch drush cron on fellowings
72 68 Florent Torregrosa
45 17 * * *  /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.ginfo > /dev/null 2>&1
73
*/15 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.clubfinance > /dev/null 2>&1
74
45 3 * * *   /users/guest/assos/bin/drushcronone default > /dev/null 2>&1
75
35 00 * * *  /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.cac13 > /dev/null 2>&1
76
*/45 * * * * /users/guest/assos/bin/drushcronone assos.centrale-marseille.fr.expat > /dev/null 2>&1
77
*/10 * * * * /users/guest/assos/bin/drushcronone forum.centrale-marseille.fr > /dev/null 2>&1
78 67 Julien Enselme
79 1 Julien Enselme
00 22 * * 7 /users/guest/assos/bin/disk-usage.sh | mail -s "disk-usage.sh" assos 
80
</code>
81 25 Florent Torregrosa
</pre>
82 1 Julien Enselme
83 67 Julien Enselme
La liste des tâches effectuées régulièrement est ci-dessous.
84 3 Julien Enselme
85 25 Florent Torregrosa
h2. La réinitialisation des variables
86
87 1 Julien Enselme
Cette tâche consiste à réinitialiser certaines variables qui donnent des droits considérés comme trop permissifs donc dangereux aux administrateurs des sites ou nécessaire pour d'autres scripts tels que piwik ou security_review.
88
89 25 Florent Torregrosa
En voici quelques exemples :
90 1 Julien Enselme
91
<pre>
92
<code class="bash">
93 25 Florent Torregrosa
drush -y vset error_level 0
94
</code>
95 1 Julien Enselme
</pre>
96 21 Florent Torregrosa
97 25 Florent Torregrosa
Cette commande permet de ne pas afficher les messages d'erreurs. En effet, ils contiennent parfois des informations sensibles sur l'installation et ne doivent donc pas être divulgués à n'importe qui.
98
99 3 Julien Enselme
<pre>
100 1 Julien Enselme
<code class="php">
101
drush -y vset --always-set reverse_proxy TRUE
102 25 Florent Torregrosa
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
103 1 Julien Enselme
</code>
104 3 Julien Enselme
</pre>
105 1 Julien Enselme
106 70 Julien Enselme
Ces commandes permettent de déclarer à Drupal les serveurs proxy du CRI afin d'éviter qu'il ne répertorie tous les visiteurs comme ayant l'adresse des serveurs sus-cités. **Pas fini : voir http://assos.centrale-marseille.fr/lessive/content/t%C3%A2che/rendre-les-settingsphp-conformes-%C3%A0-la-version-712-de-drupal**
107 1 Julien Enselme
108 6 Julien Enselme
h2. La sauvegarde des bases de données
109 1 Julien Enselme
110 3 Julien Enselme
Sur les installations d6 et d7 : une fois par semaine
111 1 Julien Enselme
112 73 Julien Enselme
On utilise les scripts de sauvegardes :
113
114
* [[Scripts_et_taches_planifiees#d6-all-dump-fullsh|d6-all-dump-full.sh]]
115
* [[Scripts_et_taches_planifiees#d6-dump-individualsh|d6-dump-individual.sh]]
116
* [[Scripts_et_taches_planifiees#d7-all-dump-fullsh|d7-all-dump-full.sh]]
117
* [[Scripts_et_taches_planifiees#d7-all-dump-individualsh|d7-all-dump-individual.sh]].
118 1 Julien Enselme
119 72 Julien Enselme
h2. La purge des sauvegardes de bases de données
120 1 Julien Enselme
121
Sur les installations d6 et d7 : toutes les semaines
122
123 72 Julien Enselme
Cette tâche utilise les scripts de purges des sauvegardes ([[Scripts_et_taches_planifiees#flush-full-databases-backupssh|flush-full-databases-backups.sh]] et [[Scripts_et_taches_planifiees#flush-individual-databases-backupssh|flush-individual-databases-backups.sh]]) afin de libérer de l'espace disque en supprimant les sauvegardes de bdd les plus vieilles.
124 1 Julien Enselme
125
h1. Liste des scripts à disposition
126 21 Florent Torregrosa
127
Les scripts utilisés sont hébergés dans le répertoire bin du compte assos.
128
129 53 Julien Enselme
h2. Configuration et utilitaires
130 1 Julien Enselme
131 53 Julien Enselme
h3. scripts-config.sh
132
133
<pre>
134
<code class="php">
135
#!/bin/sh
136
137
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/users/guest/assos/bin
138
PATH=$PATH:/usr/guest/assos/bin
139
140
scripts_config='imported'
141
142
email_multi_assos="assos@centrale-marseille.fr"
143
144
dir_multi_assos='/users/guest/assos'
145
dir_scripts="$dir_multi_assos/bin"
146
dir_template="$dir_multi_assos/template"
147
148
###### Database
149
db_server="myassos.serv.int"
150
db_user="assos"
151
dir_log="$dir_multi_assos/log"
152
153
###### Backup
154
db_full_backup_number=3
155 63 Julien Enselme
db_individual_manual_backup_number=2
156 53 Julien Enselme
db_individual_auto_backup_number=3
157
158
dir_backup="$dir_multi_assos/backup"
159
dir_full_backup="$dir_backup/full"
160
dir_individual_backup="$dir_backup/individual"
161 1 Julien Enselme
dir_individual_manual_backup="$dir_individual_backup/manual"
162
dir_individual_auto_backup="$dir_individual_backup/auto"
163
164
165
###### D7 variables
166
d7_dir="$dir_multi_assos/htmltest"
167
d7_dir_sites="$d7_dir/sites"
168
169
### Log
170
d7_dir_log="$dir_log/d7"
171
172
### Backup
173
d7_dir_full_backup="$dir_full_backup/d7"
174
d7_dir_individual_manual_backup="$dir_individual_manual_backup/d7"
175
d7_dir_individual_auto_backup="$dir_individual_auto_backup/d7"
176
177
### Template
178
d7_settings_template_name="settings-D7-bddinde-template.php"
179
d7_settings_template="$dir_template/$d7_settings_template_name"
180
181
182
###### D6 variables
183
d6_dir="$dir_multi_assos/html"
184
d6_dir_sites="$d6_dir/sites"
185
186
### Backup
187
d6_dir_full_backup="$dir_full_backup/d6"
188
d6_dir_individual_manual_backup="$dir_individual_manual_backup/d6"
189
d6_dir_individual_auto_backup="$dir_individual_auto_backup/d6"
190 64 Julien Enselme
</code>
191
</pre>
192
193
h3. scripts-config-site.sh
194
195
<pre>
196
<code class="php">
197
#!/bin/sh
198
199
# Sets useful variables of a site.
200
# ARGS: site_name
201
# Requires scripts-config.sh
202
203
# Check if scripts-config.sh is imported.
204
if [ -z $scripts_config ] ; then
205
    echo "Import of scripts-config.sh required."
206
    exit 1
207
fi
208
209
if [ -z $1 ] ; then
210
    echo "This script needs a site name as argument."
211
    exit 1
212
fi
213
214
scripts_config_site='imported'
215
216
d7_site_name=$1
217
d7_site_dir="$d7_dir_sites/assos.centrale-marseille.fr.$d7_site_name"
218
d7_site_settings="$d7_site_dir/settings.php"
219 53 Julien Enselme
</code>
220
</pre>
221
222
h3. scripts-utils.sh
223
224
<pre>
225
<code class="php">
226
#!/bin/sh
227
228
# This script contains useful functions for other scripts.
229
230
# Check if scripts-config.sh is imported.
231
if [ -z $scripts_config ] ; then
232
    echo "Import of scripts-config.sh required."
233 58 Julien Enselme
    . scripts-config.sh
234 53 Julien Enselme
fi
235
236
scripts_utils='imported'
237
238
ask_password() {
239
    # read -s doesn't work with sh.
240
    # usage: pass=`ask_password "password please:"`
241
    echo $1 >&2
242
    echo -n ">" >&2
243
    stty_avant=`stty -g`
244
    stty -echo
245
    read password
246
    stty $stty_avant
247
    echo "$password"
248
    unset password
249
}
250
251
ask_password_db() {
252
    # ARGS: server_name, user_name
253
    local db_password="pour_boucler"
254
    # empty db request to validate password
255
    while ! mysql -h $1 -u $2 -p$db_password -e "" 2>/dev/null ; do
256
        db_password=`ask_password "database password:"`
257
    done
258
    echo $db_password
259
}
260
261
generate_password(){
262
    # ARGS: [password_length]
263
    # The password contains special characters. '/' must be excluded to avoid sed malfunction.
264
265
    local site_password='/'
266
267
    if [ -z $1 ] ; then
268
        local password_length=20
269
    else
270
        local password_length=$1
271
    fi
272
273
    while echo "$site_password" | grep -Fq '/' ; do
274
        site_password=`dd if=/dev/urandom count=1 | uuencode -m - | head -n 2 | tail -n 1 | cut -c-$password_length`
275
    done
276
277
    echo $site_password
278
}
279
280
count_d7_sites(){
281
    find $d7_dir_sites -type d ! -name all -maxdepth 1 | wc -l
282
}
283
284
check_arguments() {
285 65 Julien Enselme
    # ARGS: number of arguments passed to script, number of arguments required, [help text]
286 53 Julien Enselme
    if [ $1 -lt $2  ] ; then
287 1 Julien Enselme
        echo "Number of arguments insuffisant."
288 65 Julien Enselme
	echo $3
289 53 Julien Enselme
        exit 1
290
    fi
291
}
292
</code>
293
</pre>
294
295
h2. Scripts généraux
296
297
h3. all-reset-permissions.sh
298
299
<pre>
300
<code class="php">
301
#!/bin/sh
302
303 21 Florent Torregrosa
. scripts-config.sh
304
305
# This script puts the correct permissions to sites folders, settings.php and scripts.
306
307
######### drupal 6
308
for dir in `find $d6_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all ! -name images ! -name languages` ; do
309
    chmod 755 $dir
310
    chmod 400 $dir/settings.php
311 35 Florent Torregrosa
done
312
313
######### drupal 7
314
for dir in `find $d7_dir_sites -type d -maxdepth 1 -mindepth 1 ! -name all` ; do
315
    chmod 755 $dir
316
    chmod 400 $dir/settings.php
317 21 Florent Torregrosa
done
318
319 1 Julien Enselme
####### bin
320 21 Florent Torregrosa
chmod -R 700 $dir_scripts
321
322
####### backup
323
chmod -R 700 $dir_backup
324
325
####### log
326
chmod -R 700 $dir_log
327
</code>
328 1 Julien Enselme
</pre>
329 21 Florent Torregrosa
330 53 Julien Enselme
h3. contrib-usage.sh
331 29 Florent Torregrosa
332 21 Florent Torregrosa
<pre>
333
<code class="php">
334
#!/bin/sh
335
336
. scripts-config.sh
337
. scripts-utils.sh
338
339 66 Julien Enselme
help="# ARGS: Drupal_version, project_status, project_name\n
340
# List sites that have the project_name with the corresponding project_status.\n
341
# project status: enabled or disabled"
342 21 Florent Torregrosa
343 66 Julien Enselme
check_arguments $# 3 "$help"
344 21 Florent Torregrosa
345
if [ $1 = d6 ] ; then
346
    cd $d6_dir_sites
347
else
348
    cd $d7_dir_sites
349
fi
350 29 Florent Torregrosa
351 21 Florent Torregrosa
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort ` ; do
352
    # List non-core projects that correspond to the status.
353
    # Keep project_name if listed.
354
    # Count line result. 0 if not listed or 1 if listed.
355 1 Julien Enselme
	# Print site_dir if listed.
356 21 Florent Torregrosa
357
    cd $dir;
358
	if [ 1 -eq `drush pml --no-core --status=$2 | grep $3 | wc -l` ] ; then
359
        echo $dir; 
360
    fi
361
    cd -
362
done
363 1 Julien Enselme
</code>
364 21 Florent Torregrosa
</pre>
365
366 53 Julien Enselme
h3. delete-temporary-files.sh
367 21 Florent Torregrosa
368 41 Florent Torregrosa
<pre>
369 21 Florent Torregrosa
<code class="php">
370 1 Julien Enselme
#!/bin/sh
371
372
. scripts-config.sh
373 53 Julien Enselme
374
find $dir_multi_assos -name '*~' -delete
375
</code>
376
</pre>
377
378
h3. disk-usage.sh
379
380
<pre>
381
<code class="php">
382
#!/bin/sh
383
384
. scripts-config.sh
385
386
echo -e "Quotas: 9.1 GB\n"
387
388
echo "Assos:"
389
du -hcs $dir_multi_assos | grep total
390
echo -e "\n"
391
392
echo "D6:"
393
cd $d6_dir_sites
394
du -h -d 1
395
echo -e "\n"
396
397
echo "D7:"
398
cd $d7_dir_sites
399
du -h -d 1
400
echo -e "\n"
401
402
echo "Logs:"
403
du -hcs $dir_log | grep total
404
echo -e "\n"
405
406
echo "Backups:"
407
cd $dir_backup
408
du -h -d 2
409
</code>
410
</pre>
411
412
h3. drushcronone
413
414
<pre>
415
<code class="php">
416
#!/bin/sh
417
418
. scripts-config.sh
419
420
if [ $# -lt 1 ]; then
421
  echo "usage: $0 <site's folder name>"
422
  exit 1
423
fi
424
425
cd "$d6_sites_dir/$*"
426
pwd;
427
drush cron;
428
</code>
429
</pre>
430
431
h3. flush-files.sh
432
433
<pre>
434
<code class="php">
435
#!/bin/sh
436
437 66 Julien Enselme
help="# ARGS: number of files to keep, [email address]\n
438
# usage: flush files from a folder except a number equal to argument."
439 53 Julien Enselme
440
# Check if scripts-config.sh is imported.
441
if [ -z $scripts_utils ] ; then
442
    echo "Import of scripts-utils.sh required."
443 57 Julien Enselme
    . scripts-utils.sh
444 53 Julien Enselme
fi
445
446 66 Julien Enselme
check_arguments $# 1 "$help"
447 53 Julien Enselme
448
backups_number=`ls | wc -l`
449 60 Julien Enselme
let "number_of_backups_to_delete = backups_number - $1" > /dev/null
450 53 Julien Enselme
451 1 Julien Enselme
if [ $number_of_backups_to_delete -gt 0 ] ; then
452
    ls | head -$number_of_backups_to_delete | xargs rm
453 53 Julien Enselme
else
454 60 Julien Enselme
    if [ ! -z $2 ] ; then
455
        dir=`pwd`
456
        echo "There are not enough files in $dir to Flush it. Check if backup script works fine." | mail -s "[db] $dir has a backup problem" $2
457
    fi
458 53 Julien Enselme
fi
459
</code>
460
</pre>
461
462
h3. flush-full-databases-backups.sh
463
464
<pre>
465
<code class="php">
466
#!/bin/sh
467
468
. scripts-config.sh
469
470
# ARGS: Drupal version
471
472
cd $dir_full_backup/$1
473
474
for dir in `ls` ; do
475
    cd $dir
476
    flush-files.sh $db_full_backup_number $email_multi_assos
477
    cd -
478
done
479
</code>
480
</pre>
481
482
h3. flush-individual-databases-backups.sh
483
484
<pre>
485
<code class="php">
486
#!/bin/sh
487
488
. scripts-config.sh
489
490
# ARGS: manual or auto, Drupal version
491
492
cd $dir_individual_backup/$1/$2
493
494
for dir in `ls` ; do
495
    cd $dir
496
    if [ $1 = 'auto' ] ; then
497
        flush-files.sh $db_individual_auto_backup_number $email_multi_assos
498
    else
499 61 Julien Enselme
        flush-files.sh $db_individual_manual_backup_number
500 53 Julien Enselme
    fi
501
    cd -
502
done
503
</code>
504
</pre>
505
506
h2. Scripts Drupal 6
507
508
h3. d6-all-drush.sh
509
510
<pre>
511
<code class="php">
512
#!/bin/sh
513
514
. scripts-config.sh
515 21 Florent Torregrosa
. scripts-utils.sh
516
517 66 Julien Enselme
help="# ARGS: drush_command"
518 21 Florent Torregrosa
519 66 Julien Enselme
check_arguments $# 1 "$help"
520 21 Florent Torregrosa
521
for dir in `find $d6_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all ! -name languages ! -name images | sort` ; do
522
    cd $dir
523
    echo $dir
524
    drush $*
525 1 Julien Enselme
done
526 21 Florent Torregrosa
</code>
527 45 Florent Torregrosa
</pre>
528 21 Florent Torregrosa
529 53 Julien Enselme
h3. d6-all-dump-full.sh
530 45 Florent Torregrosa
531 37 Florent Torregrosa
<pre>
532
<code class="php">
533
#!/bin/sh
534
535
. scripts-config.sh
536
537
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
538
539
# Dump D6 database with webassos user.
540
mysqldump --single-transaction webassos -h myweb.serv.int -u webassos --password=password > $d6_dir_full_backup/webassos/$current_date.webassos.sql
541 41 Florent Torregrosa
542 1 Julien Enselme
# Dump D6 form database with forum user.
543 37 Florent Torregrosa
mysqldump forum -h myweb.serv.int -u forum --password=password > $d6_dir_full_backup/forum/$current_date.forum.sql
544
</code>
545 39 Florent Torregrosa
</pre>
546 1 Julien Enselme
547 53 Julien Enselme
h3. d6-all-reset-variables.sh
548 21 Florent Torregrosa
549
<pre>
550
<code class="php">
551
#!/bin/sh
552
553
. scripts-config.sh
554
555
cd $d6_dir_sites
556 1 Julien Enselme
d6-all-drush.sh vset --always-set reverse_proxy TRUE
557 21 Florent Torregrosa
d6-all-drush.sh vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
558
drush @sites -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
559
</code>
560
</pre>
561
562 53 Julien Enselme
h3. d6-all-update-contrib.sh
563 21 Florent Torregrosa
564
<pre>
565
<code class="php">
566
#!/bin/sh
567
568
. scripts-config.sh
569
570
d6-all-drush.sh -y en update
571 1 Julien Enselme
# Launch cron to check for updates.
572 21 Florent Torregrosa
d6-all-drush.sh -q cron
573
574
d6-all-drush.sh cc all
575
d6-all-dump-full.sh
576
577
d6-all-drush.sh -y upc --no-core
578 1 Julien Enselme
d6-all-drush.sh -y updb
579
580
d6-all-drush.sh -y dis update
581
582
d6-all-drush.sh cron
583
</code>
584
</pre>
585 41 Florent Torregrosa
586 53 Julien Enselme
h3. d6-dump-individual.sh
587 41 Florent Torregrosa
588
<pre>
589
<code class="php">
590
#!/bin/sh
591
592
. scripts-config.sh
593 21 Florent Torregrosa
594
# ARGS: auto or manual, site_prefix
595
# WARNING : backup are regularly flushed, put your backup in a safe place.
596
597
cd $dir_individual_backup
598 1 Julien Enselme
599
#!/bin/sh
600
601
. scripts-config.sh
602 21 Florent Torregrosa
603 1 Julien Enselme
# ARGS: auto or manual, site_prefix
604 21 Florent Torregrosa
# WARNING : backup are regularly flushed, put your backup in a safe place.
605
606
cd $dir_individual_backup
607
608 41 Florent Torregrosa
#récupération des tables du site dans le fichier liste_tables.temp
609
tables='_%'
610
liste="$1$tables"
611
612 21 Florent Torregrosa
mysql -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY -BNe "show tables like '"$liste"'" webassos | tr '\r\n' ' ' > liste_tables.temp
613
614
#transformation de cette liste en une variable
615
var=$(cat liste_tables.temp)
616
617 1 Julien Enselme
#sauvegarde de toutes ces tables
618 21 Florent Torregrosa
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
619
suffixe="_dump$current_date.sql"
620
fichier="$1$suffixe"
621
622 1 Julien Enselme
if [ $1 = 'auto' ] ; then
623 21 Florent Torregrosa
    mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_auto_backup/$fichier
624
else
625 1 Julien Enselme
    mysqldump webassos -h myweb.serv.int -u webassos --password=HBVH2ljgyZCA0AP251DY $var > $d6_dir_individual_manual_backup/$fichier
626 21 Florent Torregrosa
fi
627 1 Julien Enselme
628
#suppression du fichier temporaire utilisé
629 21 Florent Torregrosa
rm liste_tables.temp
630
</code>
631
</pre>
632
633 53 Julien Enselme
h2. Scripts Drupal 7
634 30 Florent Torregrosa
635 53 Julien Enselme
h3. d7-all-drush-logged.sh
636
637 1 Julien Enselme
<pre>
638 30 Florent Torregrosa
<code class="php">
639
#!/bin/sh
640
641 21 Florent Torregrosa
. scripts-config.sh
642 30 Florent Torregrosa
. scripts-utils.sh
643
644 66 Julien Enselme
help="# ARGS: drush_command"
645 21 Florent Torregrosa
646 66 Julien Enselme
check_arguments $# 1 "$help"
647 1 Julien Enselme
648 21 Florent Torregrosa
cd $d7_dir_sites
649 1 Julien Enselme
650 21 Florent Torregrosa
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
651
    cd $dir
652
    current_date=`date "+%Y-%m-%d_%Hh:%Mm"`
653
    echo "======$current_date======" >> $d7_dir_log/$dir.log
654 1 Julien Enselme
    echo "Arguments are: $* " >> $d7_dir_log/$dir.log
655 21 Florent Torregrosa
    drush $* >> $d7_dir_log/$dir.log
656
    #To have the same caracters limit we needed 'end' that have 4 caracters, we used the Croate language for this ! LOL 
657
    echo -e "=============KRAJ=============\n" >> $d7_dir_log/$dir.log
658
    cd -
659
done
660
</code>
661
</pre>
662 41 Florent Torregrosa
663 53 Julien Enselme
h3. d7-all-drush.sh
664 21 Florent Torregrosa
665
<pre>
666
<code class="php">
667
#!/bin/sh
668
669
. scripts-config.sh
670 1 Julien Enselme
. scripts-utils.sh
671 21 Florent Torregrosa
672 66 Julien Enselme
help="# ARGS: drush_command"
673 21 Florent Torregrosa
674 66 Julien Enselme
check_arguments $# 1 "$help"
675 21 Florent Torregrosa
676
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
677
    cd $dir
678
    echo $dir
679
    drush $*
680
done
681
</code>
682
</pre>
683
684 53 Julien Enselme
h3. d7-all-dump-full.sh
685 21 Florent Torregrosa
686 45 Florent Torregrosa
<pre>
687 21 Florent Torregrosa
<code class="php">
688 1 Julien Enselme
#!/bin/sh
689 21 Florent Torregrosa
690
. scripts-config.sh
691
692
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
693
694
# Dump D7 databases at once.
695
mysqldump -h $db_server -u $db_user -ppassword --all-databases > $d7_dir_full_backup/myassos/$current_date.d7_full.sql
696 41 Florent Torregrosa
</code>
697
</pre>
698
699 53 Julien Enselme
h3. d7-all-dump-individual.sh
700 21 Florent Torregrosa
701
<pre>
702 41 Florent Torregrosa
<code class="php">
703
#!/bin/sh
704
705 21 Florent Torregrosa
. scripts-config.sh
706 62 Julien Enselme
. scripts-utils.sh
707 21 Florent Torregrosa
708 66 Julien Enselme
help="# ARGS: auto or manual"
709 1 Julien Enselme
710 66 Julien Enselme
check_arguments $# 1 "$help"
711 21 Florent Torregrosa
712
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
713
714
cd $d7_dir_sites
715
716
for dir in `find . -maxdepth 1 -mindepth 1 -type d ! -name all | cut -c3-` ; do
717 1 Julien Enselme
    cd $dir
718 21 Florent Torregrosa
    drush cc all
719
    if [ $1 = 'auto' ] ; then
720
        drush sql-dump --result-file="$d7_dir_individual_auto_backup/$dir/$current_date.$dir.sql"
721
    else
722 1 Julien Enselme
        drush sql-dump --result-file="$d7_dir_individual_manual_backup/$dir/$current_date.$dir.sql"
723 21 Florent Torregrosa
    fi
724
    cd -
725
done
726
</code>
727
</pre>
728
729 53 Julien Enselme
h3. d7-all-releases-info.sh
730 21 Florent Torregrosa
731
<pre>
732
<code class="php">
733
#!/bin/sh
734 1 Julien Enselme
735
. scripts-config.sh
736
737
# List version of a module or theme that exists in sites/all/*
738
# and all versions that are available for same project on drupal.org
739
740
# modules
741
cd $d7_dir_sites/all/modules
742
for dir in `ls -1` ; do
743
    if [ -d $dir ] ; then
744
        drush pm-releases $dir
745
    fi
746
done
747
748
# themes
749
cd $d7_dir_sites/all/themes
750
for dir in `ls -1` ; do
751
    if [ -d $dir ] ; then
752
        drush pm-releases $dir
753
    fi
754 21 Florent Torregrosa
done
755
</code>
756
</pre>
757 41 Florent Torregrosa
758 53 Julien Enselme
h3. d7-all-reset-variables.sh
759 41 Florent Torregrosa
760 1 Julien Enselme
<pre>
761 41 Florent Torregrosa
<code class="php">
762
#!/bin/sh
763
764
. scripts-config.sh
765
766
for dir in `find $d7_dir_sites -maxdepth 1 -mindepth 1 -type d ! -name all | sort` ; do
767
    cd $dir
768
    echo $dir
769
    d7-reset-variables.sh
770
done
771
</code>
772 1 Julien Enselme
</pre>
773 41 Florent Torregrosa
774 53 Julien Enselme
h3. d7-all-security-review.sh
775 41 Florent Torregrosa
776
<pre>
777
<code class="php">
778 1 Julien Enselme
#!/bin/sh
779 21 Florent Torregrosa
780
. scripts-config.sh
781
782
# grep -v success to keep site_name and error lines.
783 59 Florent Torregrosa
bash -c "d7-all-drush.sh security-review --store |& grep -v success | mail -s d7-all-security-review.sh $email_multi_assos"
784 21 Florent Torregrosa
</code>
785
</pre>
786
787 53 Julien Enselme
h3. d7-all-update-contrib.sh
788 21 Florent Torregrosa
789
<pre>
790
<code class="php">
791
#!/bin/sh
792
793
. scripts-config.sh
794 1 Julien Enselme
795 21 Florent Torregrosa
d7-all-drush.sh -y en update
796
# Launch cron to check for updates.
797
d7-all-drush.sh -q cron
798
799
d7-all-dump-individual.sh auto
800
801 1 Julien Enselme
d7-all-drush-logged.sh -y upc --no-core
802 21 Florent Torregrosa
d7-all-drush.sh -y updb
803
804
d7-all-drush.sh -y dis update
805
806
d7-all-drush.sh cron
807
</code>
808 1 Julien Enselme
</pre>
809 21 Florent Torregrosa
810 53 Julien Enselme
h3. d7-all-update-localisation.sh
811 21 Florent Torregrosa
812
<pre>
813
<code class="php">
814
#!/bin/sh
815
816
. scripts-config.sh
817
818
d7-all-drush.sh -y en l10n_update
819
d7-all-drush.sh l10n-update-refresh
820
d7-all-drush.sh l10n-update
821
d7-all-drush.sh -y dis l10n_update
822
</code>
823
</pre>
824 41 Florent Torregrosa
825 53 Julien Enselme
h3. d7-create-site.sh
826 1 Julien Enselme
827 21 Florent Torregrosa
<pre>
828
<code class="php">
829
#!/bin/sh
830 1 Julien Enselme
831 21 Florent Torregrosa
. scripts-config.sh
832
. scripts-config-site.sh $1
833
. scripts-utils.sh
834
835 66 Julien Enselme
help="# ARGS: site name"
836 21 Florent Torregrosa
837
######## Exceptions
838 66 Julien Enselme
check_arguments $# 1 "$help"
839 21 Florent Torregrosa
840
# "-" is forbidden because it provokes database error.
841
if [ `echo $1 | grep -` ] ; then
842
    echo '"-" is forbidden in the site name'
843
    exit 1
844
fi
845
846
# Site name length must be lower or equal to 16 due to database limitations.
847
if [`echo $1 | wc -c` -gt 16 ] ; then
848 41 Florent Torregrosa
    echo "site name can't have more than 16 characters"
849 21 Florent Torregrosa
    exit 1
850
fi
851
852 1 Julien Enselme
###### Initialisation
853 21 Florent Torregrosa
cd $d7_dir
854
db_password=`ask_password_db $db_server $db_user`
855
site_password=`generate_password`
856
857
# Check if site database already exists.
858
if mysql -h $db_server -u $db_user -e "USE $d7_site_name" -p$db_password 2>/dev/null ; then
859
    echo "Database $d7_site_name already exists"
860
    exit 1
861
fi
862
863
# Check if site folder already exists.
864
if [ -d $d7_site_dir ] ; then
865
    echo "Folder $d7_site_dir already exists"
866
    exit 1
867
fi
868
869
###### Main
870
mkdir $d7_site_dir
871
dir_site_name="assos.centrale-marseille.fr.$d7_site_name"
872
873
# Backup requirements
874 1 Julien Enselme
mkdir $d7_dir_individual_auto_backup/$dir_site_name
875 21 Florent Torregrosa
mkdir $d7_dir_individual_manual_backup/$dir_site_name
876
current_date=`date "+%Y-%m-%d-%Hh%Mm%Ss"`
877
878
# NB : ls sort by considering the 1st characters
879
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql
880
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql2
881
touch $d7_dir_individual_auto_backup/$dir_site_name/$current_date.$dir_site_name.sql3
882 1 Julien Enselme
883 21 Florent Torregrosa
# Create and grant privileges on database
884
mysql -h $db_server -u $db_user -e "CREATE DATABASE $d7_site_name" -p$db_password
885
mysql -h $db_server -u $db_user -e "GRANT ALL PRIVILEGES ON $d7_site_name.* TO '$d7_site_name'@'%' IDENTIFIED BY '$site_password'" -p$db_password
886
887
# Create settings.php
888
sed "s/\%\%DBUSER\%\%/$d7_site_name/ ; s/\%\%DBNAME\%\%/$d7_site_name/ ; s/\%\%DBPASS\%\%/$site_password/ ; s/\%\%nomsite\%\%/$d7_site_name/" < $d7_settings_template > $d7_site_settings
889
890
# Create symbolic link
891
cd $d7_dir
892
ln -s . $d7_site_name
893
894
# Next Instructions
895
echo "Go to http://assos.centrale-marseille.fr/$d7_site_name/install.php to continue."
896
echo "Press enter when ready to go on."
897 1 Julien Enselme
read key
898 41 Florent Torregrosa
899 21 Florent Torregrosa
# Init variables
900
d7-reset-variables.sh $d7_site_name
901
902
# Permissions
903
chmod -R 755 $d7_site_dir
904
chmod 400 $d7_site_settings
905
906
# Last instructions
907
echo "Last instructions:"
908
echo "- Advice the webmaster to close account creation on the website"
909
echo "- Give the webmaster a link to the club Drupal's tutorials "
910
echo "- Create a node of type \"Site\" on default"
911
echo "- Register the webmaster on webmasters@listes.centrale-marseille.fr"
912
</code>
913
</pre>
914
915 53 Julien Enselme
h3. d7-delete-site.sh
916 21 Florent Torregrosa
917
<pre>
918
<code class="php">
919
#!/bin/sh
920
921
. scripts-config.sh
922
. scripts-config-site.sh $1
923
. scripts-utils.sh
924
925 66 Julien Enselme
help="# ARGS: site name."
926 21 Florent Torregrosa
927 66 Julien Enselme
check_arguments $# 1 "$help"
928 21 Florent Torregrosa
929
db_password=`ask_password_db $db_server $db_user`
930
931
# Delete database.
932
mysql -h $db_server -u $db_user -p$db_password -e "DROP DATABASE $d7_site_name"
933
934
# Delete MYSQL user.
935 1 Julien Enselme
mysql -h $db_server -u $db_user -p$db_password -e "DROP USER '$d7_site_name'@'%'"
936 21 Florent Torregrosa
937
# Delete site's folder.
938
chmod -R 700 $d7_site_dir
939
rm -r $d7_site_dir
940
941 1 Julien Enselme
# Delete symbolic link.
942 21 Florent Torregrosa
rm $d7_dir/$d7_site_name
943
944
# Delete database backups.
945 1 Julien Enselme
rm -r $d7_dir_individual_auto_backup/assos.centrale-marseille.fr.$d7_site_name
946
rm -r $d7_dir_individual_manual_backup/assos.centrale-marseille.fr.$d7_site_name
947 21 Florent Torregrosa
948
echo "Don't forget to:"
949
echo "- Refresh node site on default"
950
echo "- Check particular behavior (normally detailed on site node)"
951
</code>
952
</pre>
953
954 53 Julien Enselme
h3. d7-reset-variables.sh
955 1 Julien Enselme
956 21 Florent Torregrosa
<pre>
957
<code class="php">
958
#!/bin/sh
959
960
. scripts-config.sh
961
962
# ARGS: [site_name]
963
964
if [ ! -z $1 ] ; then
965
    . scripts-config-site.sh $1
966
    cd $d7_site_dir
967
fi
968 1 Julien Enselme
969 41 Florent Torregrosa
current_timestamp=`date "+%s"`
970 21 Florent Torregrosa
971 41 Florent Torregrosa
random_1=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
972
random_2=`dd if=/dev/urandom count=1 2> /dev/null | cksum | cut -f1 -d" "`
973 21 Florent Torregrosa
974
####### General variables
975
drush -y vset --always-set reverse_proxy TRUE
976
drush -y vset --always-set --format=json reverse_proxy_addresses '["147.94.19.16","147.94.19.17"]'
977
drush -y ev "variable_set('update_notify_emails', array('@email_multi_assos'));"
978
drush -y vset error_level 0
979
drush -y vset dblog_row_limit 1000
980
981 1 Julien Enselme
###### Hidden captcha
982 41 Florent Torregrosa
drush -y en hidden_captcha
983 21 Florent Torregrosa
# Log wrong answers.
984
drush -y vset captcha_log_wrong_responses 1
985
# Use hidden captcha for all forms.
986
drush -y sqlq --db-prefix "UPDATE {captcha_points} SET module = 'hidden_captcha', captcha_type = 'Hidden CAPTCHA' WHERE module is NULL;"
987
# Flush captcha cache.
988
drush -y vdel captcha_placement_map_cache
989
# Randomely generate a math question as the label of the hidden captcha field.
990
drush -y vset hidden_captcha_label "$random_1 + $random_2"
991
992
993
####### Piwik
994
drush -y en piwik
995
drush -y vset piwik_site_id "101"
996 41 Florent Torregrosa
drush -y vset piwik_url_http "http://piwik.centrale-marseille.fr/"
997 21 Florent Torregrosa
drush -y vset piwik_url_https "https://piwik.centrale-marseille.fr/"
998
# Piwik cache.
999
drush -y vset piwik_cache 1
1000
drush -y vset piwik_visibility_roles "1"
1001
# Piwik is enable for everyone except the administrator
1002
drush -y vset --format=json piwik_roles '[3]'
1003
drush -y vset piwik_page_title_hierarchy 1
1004
# Activate local search.
1005
drush -y vset piwik_site_search 1
1006
1007
1008
###### Security review
1009
# For untrusted roles:
1010 41 Florent Torregrosa
# 1: anonymous user
1011 21 Florent Torregrosa
# 2: authenticated user
1012
# 3: administrator
1013
drush -y en security_review
1014 11 Julien Enselme
drush -y ev "variable_set('security_review_untrusted_roles', array('1'));"
1015 54 Florent Torregrosa
# Used to initialise entries in the database schema.
1016
drush security-review --store
1017
# Security review can't check for files permissions on multi_assos if launched within the web interface.
1018
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '1', skiptime = $current_timestamp, skipuid = '1' WHERE reviewcheck IN ('file_perms');"
1019
drush -y sqlq --db-prefix "UPDATE {security_review} SET skip = '0', skiptime = '0', skipuid = NULL WHERE reviewcheck NOT IN ('file_perms');"
1020 1 Julien Enselme
</code>
1021
</pre>